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ABSTRACT 


Several  Department  of  Defense  agencies  are  currently  investigating  the  use  of 
distributed  collaborative  virtual  environments  (CVE)  for  the  training  of  small  dismounted 
infantry  teams.  If  these  systems  are  to  be  successful,  they  will  have  to  do  more  than 
simply  allow  the  team  members  to  execute  a  task.  In  addition  to  assuring  that  essential 
training  in  the  CVE  transfers  t>  the  real  task,  we  will  also  have  to  ensure  that  aspects  of 
team  organization  also  transfer.  In  particular,  we  are  investigating  whether  or  not 
predicted  emergent  leadership,  as  measured  by  standardized  personality  tests,  holds 
within  a  CVE  or  if  aspects  of  the  interface  interfere. 

For  a  given  “real-world”  task  domain  a  leader  can  be  predicted  based  on 
personality  traits  of  the  individuals  within  the  group.  The  interface  utilized  with  a  CVE 
may  adversely  affect  these  traits.  In  other  words,  predictive  measures  of  leadership  in  the 
real  world  may  not  hold  in  a  CVE. 

The  study  reported  here  will  use  this  predictability  to  identify  the  expected 
emergent  leader  within  a  group  and  determine  how  the  CVE  interface  affects  the  ability 
of  the  predicted  individual  to  emerge  as  the  leader.  It  is  theorized  that  the  limitations  of 
CVE  interfaces  (field  of  view,  realism,  etc.)  will  negatively  impact  the  transfer  of 
leadership  personality  traits  into  the  virtual  environment,  but  not  to  a  degree  that  the 
limitation  cannot  be  overcome.  These  limitations  may  impact  the  group  dynamics  and 
the  emergent  leader  may  not  necessarily  be  the  predicted  leader  by  personality  traits. 
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I.  INTRODUCTION 


A.  MOTIVATION 

In  today’s  fast-paced  world  more  people  are  dying  on  technology  to  bring  them 
together,  rather  than  actually  gathering  in  a  common  location.  Recent  technologies,  like 
high-speed  Internet  access  virtual  reality  (VR),  have  opened  new  avenues  for  social 
interactions  and  collaboration  efforts  between  remote  locations.  We  see  corporate 
executives  using  audio  and  video  conferencing  instead  of  travel.  Network  cameras  are 
populating  homes  to  keep  families  close.  The  military  is  using  shared  VR  systems  for 
training  in  lieu  of  costly  real-world  exercises.  Collaborative  Virtual  Environments  (CVE) 
are  being  used  at  research  and  design  institutions.  These  technologies  are  changing  the 
way  people  communicate,  interact,  and  may  be  altering  how  we  socialize. 

All  of  these  technologies  have  led  to  significant  research  in  how  groups  interact 
within  a  virtual  environment  (VE).  Some  of  this  research  indicates  that  the  interface,  or 
immersion  level,  users  have  when  working  in  a  distributed  VE  (i.e.  people  are  not  co¬ 
located)  can  play  a  major  role  in  how  they  interact.  One  example  suggests  that  the  user 
with  the  most  immersive  interface  will  emerge  as  the  leader  of  the  group  when  no  leader 
is  appointed.  (Slater,  Sadagic,  Usoh,  &  Schroeder,  2000)  This  result  may  have  been 
revealed  because  of  the  way  VE  interfaces  affect  a  user’s  level  of  immersion  and 
presence  within  the  VE.  Presence  and  immersion  are  recognized  as  tightly  coupled 
concepts  that  cannot  always  be  well  defined  or  measured.  Slater,  and  others,  suggest  that 
presence  is  a  psychophysical  property  that  is  user  dependent  and  immersion  is 
technological.  (Slater  &  Wilbur,  1997)  For  example,  the  immersive  qualities  of  a  VE  are 
tied  to  physical  elements  of  the  system  like  field  of  view;  whereas,  presence  may  be  tied 
to  how  realistic  the  images  appear  to  the  user.  Previous  group  interaction  research 
conducted  by  Slater  focused  on  the  interaction  of  group  members  conducting  a 
collaboration  task  using  different  interfaces  and  thus  different  immersion  levels.  This 
study  also  focused  on  distributed  group  collaboration  when  users  have  different 
immersion  levels  to  readdress  Slater’s  research  and  to  extend  the  dynamics  of  personality 
and  group  dynamics. 
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Hie  dynamics  involved  during  group  collaboration  work  often  have  a  significant 
effect  on  the  outcome  of  the  work  produced.  The  movie,  “12  Angry  Men”  (Lumet, 
1957),  is  a  striking  example  of  how  a  person’s  personality,  and  how  they  communicate, 
can  impact  a  group’s  collaboration  effort.  The  film  does  a  good  job  of  demonstrating  the 
dynamics  of  group  deliberations  during  a  jury  trial,  the  personality  conflicts,  the  joint 
effort,  and  the  functioning  of  several  minds  working  together  to  find  the  truth.  Taking  a 
reverse  perspective,  it  is  easily  seen  how  the  dynamics  of  a  group  can  impact  any  single 
participant.  This  social  dynamic  is  often  termed  the  “Abilene  Paradox.”  This  condition 
exists  when  the  members  of  a  group  succumb  to  a  common  goal  that  is  contrary  to  each 
individual’s  goal.  As  an  example,  imagine  the  result  when  one  member  of  a  group  makes 
a  suggestion  they  assume  is  the  goal  of  the  other  group  members,  but  different  than  their 
own  desires.  Now,  the  other  group  members  assume  the  suggested  goal  meets  the 
person’s  desires  and  they  each  agree  to  the  suggestion.  They  agree  even  though  the  goal 
is  not  their  own  personal  desire.  The  result  is  a  group  that  works  to  achieve  a  goal  that  no 
single  member  desires. 

There  are  many  other  factors  involved  in  the  dynamics  of  a  group  beyond  those 
mentioned  above.  As  an  example,  some  research  indicates  that  an  attribute  like  gender 
can  have  an  impact  on  group  dynamics  (Kimble  &  McNeese,  1987).  It  gets  more 
complicated  when  variations  on  these  factors  are  considered:  group  gender  mix,  leader 
gender,  etc.  Given  the  complexities  of  group  dynamics  and  the  amount  of  research  that 
exists,  shouldn’t  equal  consideration  be  given  to  the  complexities  of  group  dynamics  in 
virtual  environments?  Specific  attention  must  be  paid  to  groups  that  collaborate  within  a 
shared  virtual  environment  when  the  participants  are  not  co-located. 

B.  OBJECTIVE 

This  study  was  designed  to  whittle  away  at  some  of  the  unknown  qualities  of 
collaboration  within  distributed  virtual  environments.  To  this  end,  the  study  looked 
specifically  at  the  relationship  between  system  interface,  personality  traits,  and  emergent 
leadership  within  a  distributed  collaborative  virtual  environment. 

It  is  widely  known  in  social  psychology  circles  that  certain  personality  trait 
combinations  are  associated  with  emergent  leadership  in  group  interactions.  These  trait 
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combinations  are  highly  dependent  on  the  task  domain.  For  instance,  the  traits  required 
for  emergent  leadership  within  a  social  domain  may  not  be  the  same  as  those  within  a 
particular  work  domain.  Emergent  leadership  traits  vary  across  different  task  domains, 
but  are  constant  within  that  task  domain.  For  a  given  domain,  a  leader  can  be  predicted 
based  on  personality  traits  of  the  individuals  within  the  group.  To  illustrate,  if  a  group 
member  exhibits  traits  associated  with  high  verbalization  and  directive  skills,  they  can  be 
predicted  with  confidence  to  emerge  as  the  leader  of  a  group  containing  members  with 
lower  trait  scores  in  those  areas.  This  assumes  the  traits  exhibited  are  associated  with  the 
task  domain.  (Schultz,  1990;  Kimble  &  McNeese,  1987;  Knowlton  &  McGee,  1994) 

This  study  used  this  predictability  to  identify  the  expected  leader  within  a  group 
and  then  test  the  effect  of  the  CVE  interface  on  the  ability  of  the  selected  individual  to 
emerge  as  the  leader.  It  was  theorized  that  the  limitations  of  CVE  interfaces  (field  of 
view,  degree  of  realism,  etc.)  would  negatively  impact  the  transfer  of  leadership 
personality  traits  into  the  virtual  environment,  but  not  to  a  degree  that  these  limitations 
could  not  be  overcome.  These  limitations  may  impact  the  group  dynamics  and  the 
emergent  leader  may  not  necessarily  be  the  predicted  leader  by  personality  trait. 

C.  APPROACH 

To  achieve  this  objective  this  study  took  the  approach  of  designing  a  CVE  that 
allowed  a  group  to  conduct  a  collaboration  task  that  hey  were  familiar  with.  The  task 
used  was  a  navigation  task  within  a  natural  tenain  domain.  Specifically,  the  participants 
were  required  to  search,  navigate,  identify,  and  locate  four  targets  within  a  virtual  model 
of  a  natural  outdoor  scene. 

Hie  virtual  model  replicated  a  section  of  land  at  the  former  Fort  Ord  military 
installation  in  Seaside,  California.  This  model  was  used  because  it  provided  the  ability  to 
replicate  as  closely  as  possible  a  real  world  scene  so  that  comparisons  could  be  made 
with  future  work  between  group  collaboration  within  a  CVE  and  the  “real-world”  setting. 
Each  participant  completed  a  questionnaire  to  define  two  main  characteristics  associated 
with  this  study:  personality  and  task  expertise.  A  personality  profile  of  the  individual 
was  recorded  along  with  the  individual’s  self-rating  of  their  expertise  on  the  task.  These 
two  criteria  were  used  to  group  the  individuals  for  the  study.  Specifically,  the  groups 
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were  arranged  so  that  one  participant  could  be  predicted  to  emerge  as  the  leader  if  the 
same  task  was  conducted  in  the  real  world. 

During  and  after  the  experiment  the  participants  were  evaluated  regarding  the 
role  they  played  within  the  group.  Measures  were  taken  to  evaluate  who  in  the  group 
emerged  as  the  leader.  The  emergent  leader  participant’s  leadership  and  personality  trait 
scores  were  analyzed  for  correlations  with  ratings  of  CVE  interface  devices  to  develop 
results. 
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H.  BACKGROUND 


A.  THE  FIVE  FACTOR  MODEL  OF  PERSONALITY 

Several  different  personality  measurement  scales  can  be  used  to  explain  an 
individual’s  personality.  Some  of  the  most  popular  are  the  Minnesota  Multiphasic 
Personality  Inventory  (MMPI),  the  Myers-Briggs  Type  Indicator  (MBIT),  and  the  Five- 
Factor  Model  (FFM).  Each  tool  has  its  own  advantages/disadvantages  and  proponents. 
This  study  used  a  variation  of  the  FFM  called  the  NEO  Five -Factor  Inventory  (NEO- 
FFI).  The  NEO-FFT  is  actually  a  reduced  version  of  the  revised  NEO  Personality 
Inventory  (NEO  PI-R),  which  has  its  roots  in  the  FFM. 

The  “FFM  originated  in  initial  works  by  Fiske  (1949),  Norman  (1963),  and 
Tuppes  and  Christal  (1963),  who  produced  a  highly  stable  structure  with  five  factors” 
(Salgado,  1997,  p.  30).  While  most  models  are  derived  from  theoretical  perspectives,  the 
FFM  has  a  theoretically  neutral  position  (Widiger,  1997).  The  NEO  PI-R  is  a  widely 
accepted  measure  of  personality  developed  by  Costa  and  McCrae,  and  assesses 
personality  in  terms  of  Neuroticism,  Extraversion,  Openness,  Agreeableness,  and 
Conscientiousness.  The  five  personality  factors  are  described  in  the  following  way: 

People  who  score  high  on  NEUROTICISM  typically  report  negative  emotions 
such  as  worry,  insecurity,  self-consciousness,  and  tempermentalness  (McCrae  & 

Costa,  1987)  whereas  people  with  low  Neuroticism  are  calm,  self-confident,  and 
cool  (Salgado,  1997). 


EXTRA  VERSION  is  the  factor  that  describes  people  who  are  rated  by  their  peers 
as  “sociable,  fun- loving,  affectionate,  friendly,  and  talkative”  (McCrae  &  Costa, 
1987,  p.  87)  versus  “reserved,  timid,  and  quiet”  (Salgado,  1997,  p.  30). 

The  final  factor  in  this  model  is  OPENNESS.  Adjectives  from  lexical  studies  that 
describe  this  factor  include  “original,  imaginative,  broad  interests,  and  daring” 
(McCrae  &  Costa,  1987,  p.  87).  “Openness  defines  individuals  who  are  creative, 
curious,  and  cultured  versus  practical  with  narrow  interests.”  (Salgado,  1997,  p. 
30) 
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People  high  in  AGREEABLENESS  are  forgiving,  lenient,  sympathetic,  agreeable, 
and  softhearted,  according  to  peer  ratings  (McCrae  &  Costa,  1987).  Peers 
describe  those  low  in  Agreeableness  in  more  negative  terms:  ruthless, 
uncooperative,  suspicious,  and  stingy. 

Peers  describe  people  high  in  CONSCIENTIOUSNESS  as  careful,  well 
organized,  punctual,  ambitious,  and  persevering  (McCrae  &  Costa,  1987). 
Conscientiousness  includes  both  proactive  (hardworking,  ambitious)  and 
inhibitive  (dutiful,  scmpulous)  aspects  (McCrae  &  Costa,  1989). 

These  personality  measurement  tools  are  important  because  they  can  be  used  to 
closely  predict  which  individual  should  emerge  as  the  leader  for  a  given  group 
collaboration  task. 

B.  PERSONALITY  MEASURES 

Some  personality  measurement  techniques  are  more  objective  while  others  are 
subjective  and  prone  to  bias.  A  goal  for  this  study  was  to  use  a  technique  that  is 
standardized,  and  that  also  has  high  reliability,  and  validity.  These  attributes  tend  to 
define  the  best  assessment  techniques.  Standardization,  reliability,  and  validity  ensure 
consistency  for  administration,  consistency  of  results,  and  accuracy  of  measurement.  The 
NEO-FFI  has  these  qualities.  As  an  example,  Salgado  (1997)  while  evaluating  the 
validity  and  consistency  of  various  personality  measures  for  predicting  job  performance 
in  the  European  community  confirmed  that  tests  based  on  the  Five -factor  Model  provide 
consistent  results  across  both  North  American  and  European  communities.  The  NEO  PI- 
R  was  among  the  tests  used  and  the  NEO-FFI  is  fully  based  on  the  NEO  PI-R.  McCrae 
and  Costa  (1987)  also  compared  the  NEO  PI-R  measures  between  self-reports  and  peer 
ratings  and  found  statistically  significant  validation  of  the  measures. 

A  primary  method  for  assessment  is  the  self-report  inventory  method.  The  self- 
report  method  asks  people  to  report  on  themselves  by  answering  questions  about  their 
feelings  and  behavior  in  a  variety  of  situations.  The  person  taking  the  survey  must 
indicate  how  closely  each  item  describes  their  own  traits  or  how  much  they  agree  with 
each  item.  The  standardized  administration,  scoring,  and  evaluation  of  the  NEO-FFI 
allows  for  effective  self-reporting. 
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As  indicated  above,  the  MBTI  and  MMPI  are  alternatives  for  personality 
assessment.  The  MBTI  requires  several  hours  to  administer  and  evaluate  and  requires  a 
trained  psychological  professional  to  interpret  the  scores.  The  MMPI  determines 
personality  traits  of  hypochondriasis,  depression,  hysteria,  psychopathic  deviate, 
masculinity- feminity,  paranoia,  psychasthenia,  schizophrenia,  hypomania,  and  social 
introversion.  The  MMPI  is  primarily  used  by  clinical  psychologists  as  a  diagnostic  tool 
for  assessing  personality  disorders,  but  is  also  utilized  as  a  vocational  tool.  Like  the 
MBTI,  the  MMPI  is  extremely  long  to  administer  and  require  special  training  to  interpret 
the  results. 

The  NEO-FFI,  on  the  other  hand,  is  easy  to  administer  and  interpret.  The  entire 
process  can  take  less  than  60  minutes  per  individual.  It  has  been  validated  and  used  in 
previous  research  at  the  Naval  Postgraduate  School.  For  these  reasons,  the  NEO-FFI  fits 
the  requirements  of  this  study  well. 
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III.  APPARATUS 


A.  NEO-FFI 

This  study  required  an  assessment  tool  that  has  high  reliability  and  validity 
ratings,  while  also  being  easy  to  administer,  easy  to  complete  with  minimal  time 
requirements,  and  easy  to  interpret.  The  NEO  PI-R,  the  predominant  measure  of  the  five- 
factor  model  of  personality,  was  chosen  for  inventorying  an  individual’s  personality  since 
it  meets  all  the  above  criteria.  (Widiger,  1997)  The  NEO  PI-R  consists  of  240  statements 
to  which  a  person  indicates  their  degree  of  agreement  on  a  5- point  scale.  The  NEO  PI-R 
is  often  referred  to  as  a  lexical  five- factor  model  since  it  attempts  to  define  personality  in 
natural  language  terms. 

Buziak  (2000,  p.  24)  states  that  “substantial  research  exists  regarding  NEO  PI-R 
reliability  and  validity.  Most  importantly,  the  NEO  PI-R  has  demonstrated  consistent 
convergent  and  discriminant  validity,  as  well  as  indicating  how  alternate  models  can  be 
understood  from  the  perspective  of  the  five  factor  model.”  He  goes  cn  to  note  that  many 
studies  on  personality  measurement,  including  the  MBTI,  have  used  the  NEO  PI-R  as  a 
comparison  to  find  overlaps  because  of  the  consistency  in  results.  The  use  of  the  NEO 
PI-R  for  comparison  is  due  to  the  high  correlation  with  the  five  factor  model,  which  does 
not  rely  on  a  particular  theory  of  personality.  (McCrae  &  Costa,  1987) 

The  NEO  Five  Factor  Inventory  (NEO-FFI)  is  a  60- question  subset  of  the  full 
240-question  NEO  PI-R.  The  NEO  PI-R’s  additional  length  allows  for  more  precise 
measurement  and  better  false  answer  detection  while  the  shorter  length  of  the  NEO-FFI 
required  less  time  to  administer.  The  authors  of  the  NEO  PI-R  do  not  envision  any 
significant  changes  in  the  near  future;  thus,  it  is  a  logical  conclusion  that  there  will  be  no 
major  revisions  planned  for  the  NEO-FFI  and  it  can  be  used  with  confidence.  (McCrae  & 
Costa,  1992) 

B.  TASK  EXPERTISE 

Establishing  a  profile  of  the  individual  personalities  alone  was  not  sufficient  for 
this  study  of  emergent  leadership.  As  indicated  above,  specific  personality  traits  are 
often  effective  within  certain  task  domains.  Since  this  study  used  a  land  navigation 

9 


exercise  for  the  collaborative  task,  each  individual’s  expertise  and  experience  within  this 
task  domain  was  profiled.  As  part  of  developing  these  profiles,  a  land  navigation 
background  questionnaire  was  developed  and  utilized. 

The  questionnaire  defined  broad  categories  of  expertise  levels  within  the  land 
navigation  domain.  By  using  a  profile  of  each  individual,  the  groups  could  be  formed  so 
that  no  single  individual  had  a  significant  advantage  on  the  task  based  solely  on  their  land 
navigation  expertise.  This  was  important  in  order  to  ensure  personality  traits  were  the 
underlying  force  behind  any  emergent  leadership.  Expertise  of  a  group  leader  plays  an 
important  role  in  land  navigation  tasks  because  the  leader  is  looked  upon  for  guidance 
and  training  (Stine,  2000). 

C.  VIRTUAL  ENVIRONMENT  MODEL 

A  distributed  collaborative  virtual  environment  was  developed  to  support  the 
collaborative  task  for  this  study.  The  distributed  nature  of  the  CVE  presented  a 
significant  human  factors  issue  in  the  experiments  conducted  by  Slater  (2000).  To 
support  this  endeavor,  the  virtual  model  was  represented  on  three  high-end  graphics 
computers  over  a  Local  Area  Network  (LAN).  Each  computer  interface  provided  a 
different  degeree  of  immersion  similar  to  the  Slater  experiments.  The  individual 
immersive  interfaces  utilized  a  CAVE  (CAVE  Automatic  Virtual  Environment  - 
originally  developed  at  University  of  Illinois,  Chicago)  interface,  a  system  with  three 
large- screen  monitors,  and  a  standard  computer  monitor.  All  other  interface  tools,  such 
as  input  devices,  were  kept  identical. 

1.  Virtual  Model  Description 

The  shared  virtual  environment  was  created  using  Multigen- Paradigm  Creator. 
The  entire  model  was  built  to  scale  with  overall  dimensions  of  approximately  1000  by 
500  meters.  Terrain  elevation  was  developed  in  accordance  with  local  topographic  maps 
having  contours  every  ten  feet.  A  satellite  image  of  the  area  was  overlaid  on  the  terrain 
map  and  helped  to  establish  paths  within  the  environment.  Photographs  of  objects  and 
vegetation  were  taken  of  the  represented  area  and  converted  to  objects  and  placed  in  the 
virtual  model.  Figure  1  shows  a  sample  view  of  the  virtual  model,  as  the  user  would  have 
experienced  it.  The  end  result  of  the  model  was  an  environment  where  the  user,  while 
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moving,  experienced  parallax  and  obscured  visibility  due  to  vegetation.  Additionally, 
they  could  identify  key  landscape  features,  terrain  elevation  changes,  and  paths. 


Figure  1.  Virtual  Model  as  Seen  by  User. 

2.  High-Immersion  Station 

The  workstation  used  by  the  participants  that  provided  the  highest  level  of 
immersion  was  implemented  using  a  CAVE-like  interface.  It  was  composed  of  three  (3) 
large  rear  projection  displays  each  having  an  81 -inch  diagonal  measurement  as  depicted 
in  Figure  2.  The  displays  were  connected  together  at  a  near  seamless  45  degrees  and 
almost  completely  filled  the  user’s  normal  field  of  view.  The  virtual  model  was  rendered 
in  real-time  via  Multigen- Paradigm’s  Vega  software.  The  view  point  for  each  display 
was  rendered  using  three  45-degree  frustrums  set  at  45-degree  offsets,  which  together 
presented  a  135 -degree  graphical  field  of  view. 
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Figure  2.  High- Immersive  Display  Layout. 


Each  display  projector  was  driven  by  a  dedicated  high-end  personal  computer 
(PC).  The  computers  utilized  the  Vega  software  feature  that  allows  one  “master” 
computer  to  distribute  scene  rendering  to  multiple  “slave”  computers.  All  three 
computers  were  linked  via  a  high-speed  gigabit  network  and  the  master  controlled  the 
display  synchronization.  Table  1  shows  the  hardware  configuration  for  each  computer. 


Operating  System 

Windows  2000  (SP2) 

Computer  Processor 

Dual  Intel  Pentium  HI,  lGhz 

Memory 

2  Gigabytes 

Graphics  Adapter 

Nvidia  GeForce  3 

Frames  per  Second 

Greater  than  25 

Table  1.  High- Immersion  Workstation  PC  Configuration. 
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3. 


Mid-Immersion  Station 


The  mid- level  immersive  station  was  rendered  using  the  same  software 
configuration  as  the  high- immersion  station;  however,  at  this  station  the  computer  and 
display  device  were  significantly  different.  This  station  utilized  a  single  PC  with  multiple 
graphics  adapters  to  render  the  scene  to  a  single  Panoram  PV230  DSK  display  that 
integrates  three  (3)  15-inch  high- definition  LCD  displays.  The  PV230  display  provided 
the  user  with  a  high  quality  semi- immersive  display  of  the  virtual  environment.  Table  2 
shows  the  hardware  configuration  for  the  PC  rendering  to  the  PV230  display. 

4.  Low-Immersion  Station 

The  low-level  immersive  station  was  rendered  using  an  Intergraph  computer  with 
multiple  graphics  adapters,  but  only  one  display  device  actually  rendered  the  virtual 
model.  Rather  than  display  three  separate  frustrums,  this  station  only  rendered  a  single 
45 -degree  frustrum  sent  to  a  single  21- inch  display  monitor.  This  configuration  provided 
the  user  with  a  significantly  reduced  graphical  and  physical  field  of  view.  Table  2  shows 
the  hardware  configuration  for  the  PC. 


Station 

Mid- Immersion  Station 

Low- Immersion  Station 

Operating  System 

Windows  2000  (SP2) 

Windows  NT 

Computer  Processor 

Dual  Intel  Pentium  II,  800  Mhz 

Dual  Intel  Pentium  II,  450  Mhz 

Memory 

512  Megabytes 

512  Megabytes 

Graphics  Adapter 

Three  3Dlabs  Oxygen  GVX1 

Wildcat  2000 

Frames  per  Second 

Approximately  10 

Approximately  20 

Table  2.  Mid-  and  Low- Immersion  Workstation  PC  Configurations. 


All  other  features  of  the  user  interface  at  each  station  were  kept  the  same.  Each 
participant  sat  at  the  workstation  and  used  a  keyboard  and  mouse  to  manipulate  the 
virtual  scene.  Each  model  was  rendered  on  the  display  at  a  high  resolution  of  1024  by 
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768  pixels.  Additional  information  on  hardware  and  software  configuration  is  located  in 
Appendix  A. 

D.  USER  REPRESENTATION  AND  CONTROLS 

As  is  normal  for  distributed  collaborative  environments,  the  participant  human- 
human  interaction  methods  are  severely  limited  compared  to  real  world  interactions. 
Slater’s  experiment  essentially  limited  the  human  interaction  to  voice  communications. 
In  comparison,  this  study  used  voice  communication  augmented  with  a  “life-like”  avatar 
with  a  limited  set  of  controls  including  a  form  of  pointing.  These  limitations  impact 
group  collaboration  effectiveness  because  people  usually  use  other  methods  of 
communicating  ideas  and  desires,  such  as  touch,  facial  gestures,  and  body  language. 

1.  Voice  Communication 

Voice  communication  between  the  participants  was  handled  by  “voice- over- IP”. 
Each  user  was  provided  an  audio  headset  with  an  attached  boom  microphone. 
Additionally,  each  headset  had  controls  to  adjust  audio  level  and  microphone  mute.  A 
freely  available  software  product,  “Team  Sound”  by  RedWired  Software,  connected  each 
user  to  a  dedicated  server.  To  reduce  network  traffic  and  keep  communication  as  natural 
as  possible,  all  transmissions  were  controlled  by  a  voice- activated  feature  of  the  software. 
For  clarity,  a  minimal  compression  algorithm  was  used.  Slight  transmission  delays  were 
noted  by  users,  although  all  easily  adapted. 

2.  Avatar  Control  and  Movement 

Each  user  interacted  with  the  virtual  model  via  an  avatar  generated  by  Boston 
Dynamics  DI-Guy  software.  This  software  provided  a  realistic  representation  of  each 
user  as  a  “soldier”  carrying  a  weapon.  The  avatar  motions  and  actions  were  controlled  by 
keyboard  commands.  Each  user  could  command  their  avatar  to  “turn  left/right,”  “stand,” 
“walk,”  “walk  backward,”  “run,”  “kneel,”  “lay  down,”  and  “aim  rifle.”  Figure  3  shows 
these  various  states.  By  commanding  the  avatar  to  “turn  left/right,”  “stand,”  “walk,” 
“walk  backward,”  or  “run”  the  user  was  able  to  navigate  through  the  virtual  environment. 
The  “kneel”  and  “lay  down”  actions  were  available  to  allow  the  user  to  change  the  height 
of  their  viewpoint.  Changing  view  height  was  helpful  to  the  user  in  areas  where  they 
wanted  to  see  under  the  low  branches  of  trees. 
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Since  this  experiment  required  the  participants  to  navigate  together,  the  capability 
of  each  user  to  be  able  to  point  at  objects  was  deemed  necessary.  The  “aim  rifle”  feature 
met  this  requirement  by  allowing  the  user  to  point  their  rifle  in  a  given  direction.  It 
should  be  noted  that  this  is  a  common  way  to  point  under  a  dismounted  infantry  military 
exercise  context. 


Figure  3.  Avatar  States. 


Hie  only  other  control  the  participants  had  available  was  mouse  control  of 
viewpoint.  The  intent  was  to  allow  the  user  to  be  able  to  look  around  without  changing 
their  direction  of  movement.  This  emulates  the  notion  of  walking  and  turning  your  head 
to  see  something  to  your  side.  The  mouse  motions  were  translated  into  viewpoint 
changes  in  azimuth  and  elevation. 
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There  were  two  negative  artifacts  of  the  avatar  that  were  introduced  as  a  result  of 
software  limitations.  The  first  being  that  the  transitions  between  avatar  states  were  not 
smooth.  For  instance,  when  a  user  commanded  the  avatar  to  “aim  rifle'’  the  action 
occurred  immediately  rather  than  affording  a  smooth  transition  of  raising  the  rifle.  The 
second  artifact  was  that  each  user  could  not  see  their  own  avatar,  with  the  exception  of 
their  rifle.  Although  seeing  their  own  rifle  was  helpful,  especially  when  trying  to  realign 
view  direction  with  movement  direction,  it  provided  very  little  proprioceptive  feedback. 

All  of  the  character  states  of  each  user  were  transmitted  to  the  other  participants 
and  the  representative  avatar  would  change  state.  Overall  this  provided  a  more  realistic 
expression  of  each  user  than  Slater’s  study,  although  still  a  ‘Tar-cry”  from  what  people 
experience  in  real-world  collaboration. 


16 


IV.  METHODOLOGY 


A.  EVALUATING  PARTICIPANT  PERSONALITY 

As  mentioned  above,  this  research  required  profiling  each  participant’s 
personality.  These  profiles  were  used  to  form  collaboration  groups  for  the  experiment 
task.  Specifically,  each  group  required  one,  and  only  one,  individual  with  personality 
traits  that  would  predict  that  this  person  would  emerge  as  the  group  leader.  The  NEO- 
FFI  provided  the  framework  for  profiling  the  potential  participant  personalities.  In 
addition  to  forming  groups  based  on  personality,  each  participant’s  expertise  on  the 
experiment  task  was  also  profiled. 

1.  Survey  Implementation 

The  questionnaires  for  profiling  personality  and  expertise  were  combined  into  one 
document.  This  helped  to  limit  the  number  of  separate  documents  the  participants  were 
required  to  complete.  Although  the  profiling  was  conducted  using  one  document,  the 
questions  were  divided  into  separate  sections  so  that  the  participant  noticed  the  shift  in 
question  domains. 

NEO-FFI  personality  surveys  from  Psychological  Assessment  Resources,  Inc. 
were  used  for  documenting  each  individual’s  personality  traits.  Because  this  study 
required  participants  that  possessed  specific  personality  traits,  a  large  pool  of  perspective 
participants  was  needed.  To  meet  this  objective,  the  NEO-FFI  survey  was  additionally 
converted  to  hypertext  documents  and  made  available  via  a  secure  web  site.  Care  was 
taken  in  developing  the  web-based  documents  to  ensure  respondents  understood  how  to 
complete  the  survey  and  their  rights  to  privacy.  For  those  individuals  who  did  not  desire 
to  complete  the  personality  survey  via  the  web,  information  was  provided  on  how  to 
complete  the  questionnaires  using  the  normal  paper  documents. 

2.  Survey  Documents 

Each  of  the  survey  implementations  presented  the  instructions  and  questions  in 
exactly  the  same  wording  and  format.  Figures  4  and  5  provide  examples  of  each  of  the 
survey  implementations.  The  only  notable  difference  between  the  implementations  was 
in  the  recording  format.  The  hypertext  document  provided  “radio  buttons”  adjacent  to 
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the  questions  and  responses  were  automatically  recorded  to  an  electronic  database.  The 
traditional  paper  documents  had  a  page  for  questions  and  a  separate  page  for  recording 
the  answers  to  the  questions.  The  electronic  database  automatically  keyed  each  user  with 
a  unique  identification  number.  This  number  was  used  throughout  the  experiment  to 
anonymously  record  information  about  each  participant.  Responses  recorded  via 
traditional  paper  documents  were  manually  entered,  by  the  principle  investigator,  into  the 
electronic  database,  and  a  unique  identification  number  was  assigned. 


NEO  Five-Factor  Inventory  (Form  S) 

SD 
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SA 
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am  not  a  worrier. 
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like  to  have  a  lot  of  people  around  me. 

3.  r 

r 
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r 

r 

i 

don’t  like  to  waste  my  time  daydreaming. 

4.  r 
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r 
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try  to  be  courteous  to  everyone  I  meet. 

5.  r 

r 

r 

r 

r 

i 

keep  my  belongings  clean  and  neat. 

Figure  4.  Question  Format  from  Web-based  Survey. 


1 .  I  am  not  a  worrier. 

2.  I  like  to  have  a  lot  of  people  around  me. 

3.  I  don’t  like  to  waste  my  time  daydreaming. 

4.  I  try  to  be  courteous  to  everyone  I  meet. 

5 .  I  keep  my  belongings  clean  and  neat. 


Enter  your  responses  here— remember  to  enter  responses  across  the  rows. 
SD  ^Strongly  Disagree;  D  =  Disagree;  N  =  Neutral;  A  =  Agree;  SA = Strongly  Agree 
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Figure  5.  Question  Format  from  Paper-based  Survey.  After  Ref.  (McCrae  &  Costa,  1992). 
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All  the  questions  on  the  survey  required  the  respondent  to  make  a  decision  on 
whether  they  agreed  with  a  particular  statement  on  a  scale  from  “Strongly  Disagree”  to 
“Strongly  Agree”.  The  NEO-FFI  converts  these  answers  to  a  numeric  value  used  for 
scoring  the  various  personality  traits.  It  was  these  numeric  values  that  were  recorded  in 
the  database  for  later  analysis. 

3.  Survey  Responses 

After  completing  the  survey,  each  participant  was  advised  they  would  be 
contacted  to  coordinate  any  further  participation.  This  allowed  time  for  the  coordinator 
to  review  the  surveys  and  establish  the  groups  for  the  experiment.  All  communication 
after  completing  the  survey  was  handled  via  email. 

A  total  of  67  people  initiated  the  survey,  but  only  56  completed  all  required 
information  and  questions.  The  56  respondents  were  all  staff  or  students  at  the  Naval 
Postgraduate  School  in  Monterey,  California.  All  respondents  were  over  the  age  of  21;  4 
of  the  56  respondents  were  female. 

4.  Emergent  Leadership  Personality  Traits 

The  traits  associated  with  emergent  leadership  for  this  study  were  related  to  the 
required  navigation  and  communication  tasks.  The  experiment  focused  primarily  on 
communication  over  navigation.  Previous  research  conducted  for  the  Office  of  Naval 
Research  (ONR),  indicated  that  certain  personality  traits  are  more  useful  for  certain  tasks. 
That  research  divided  tasks  into  broad  categories:  realistic,  investigative,  artistic,  social, 
enterprising,  and  conventional.  For  this  study,  the  required  navigational  and 
communicational  tasks  fit  into  the  realistic  and  social  domains  -  primarily  the  social 
domain.  Based  on  the  research  for  ONR,  group  performance  for  the  assigned  task  should 
improve  when  the  members  of  the  group  have  traits  that  are  high  in  conscientiousness 
and  agreeableness.  (Hogan,  Raza,  Sampson,  Miller,  &  Salas,  1989) 

A  number  of  studies  have  compared  leadership  and  personality.  In  1994,  Hogan, 
Curphy,  and  Hogan,  conducted  an  in-depth  analysis  of  these  studies  and  concluded 
emergent  leadership  is  a  specialized  niche  within  the  leadership  domain  that  develops  in 
small  groups  when  no  appointed  leader  exists.  The  result  of  the  analysis  Hogan,  et  al. 
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(1994)  conducted  on  emergent  leadership  shows  a  correlation  between  emergent 
leadership  and  personality  traits.  Specifically,  that  the  personality  traits  of  high  surgeney 
(sociability,  social  presence,  etc.),  conscientiousness,  and  emotional  stability  are 
correlated  to  emergent  leaders.  These  traits  closely  map  to  high  scores  in  extraversion 
and  conscientiousness,  and  a  low  score  in  neuroticism  on  the  NEO-FFI  personality 
assessment,  respectively.  By  combining  these  two  results,  this  study  anticipated  that 
extraverted,  conscientious,  and  emotionally  stable  people  would  emerge  as  leaders  in  a 
group  that  was  made  up  of  members  that  did  not  possess  these  traits. 

The  NEO-FFI  divides  the  scale  for  each  of  the  five  traits  into  categories:  Very 
Fow,  Fow,  Average,  High,  and  Very  High.  This  study  considered  the  terms  “low”  and 
“high”  in  previous  research  to  equate  to  Fow/Very  Fow,  and  High/Very  High, 
respectively.  Of  the  pool  of  56  respondents,  only  7  met  the  personality  characteristic  of 
“emergent  leader”;  specifically,  high  or  very  high  scores  in  extraversion  and 
conscientiousness,  and  a  low  or  very  low  score  in  neuroticism.  To  provide  a  larger  pool 
of  potential  leaders,  another  pass  was  taken  through  the  database  to  find  all  the  people 
that  did  not  meet  the  personality  characteristics,  but  differed  only  by  an  “average”  score 
in  one  of  the  evaluated  traits.  For  example,  someone  with  an  “average”  score  on 
extraversion,  high  score  on  conscientiousness,  and  low  score  on  neuroticism  would  not 
have  been  among  the  original  7,  but  would  have  been  included  in  the  second  pass.  This 
second  pass  increased  the  “potential  leader”  pool  by  12  people. 

5.  Other  Group  Member  Personalities 

Once  the  list  of  potential  leaders  had  been  generated,  another  pass  was  taken 
through  the  list  of  respondents  to  find  prospective  group  members.  On  this  pass  the 
primary  consideration  was  to  identify  those  individuals  that  had  personality  traits 
opposite  of  those  selected  for  the  “emergent  leader”  pool.  Specifically,  the  goal  was  to 
select  people  that  had  personality  traits  with  the  following  characteristics:  low 
extraversion,  low  conscientiousness,  and  high  neuroticism.  This  pass  provided  only  a 
handful  of  potential  group  members,  which  was  not  sufficient  for  this  study. 

To  broaden  the  list  of  potential  group  members  who  had  personality  traits 
opposite  of  the  traits  possessed  by  the  predicted  group  leaders,  a  list  was  generated  that 
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contained  only  people  with  average  or  lower  extraversion  scores.  Extraversion  is  the  trait 
with  the  highest  correlation  to  emergent  leadership  (Hogan,  Curphy,  &  Hogan,  1994). 
This  provided  a  pool  of  22  people  that  could  be  used  as  group  members  to  be  paired  with 
the  predicted  leaders.  Given  the  two  lists,  predicted  leaders  and  followers,  groups  were 
formed  based  on  the  available  schedules  of  the  potential  participants.  A  “best  effort”  was 
made  to  keep  the  group  members  task  expertise  at  a  common  level  so  that  any  emergent 
leader  would  be  based  solely  on  personality  attributes.  Additionally,  the  groups  were 
gender  homogenous  to  alleviate  any  gender  biasing. 

B.  COLLABORATION  TASK 

Each  group  was  assigned  a  task  that  was  designed  to  maximize  collaboration 
among  the  group  members.  A  land- navigation  task  was  devised  to  achieve  this  objective. 
Land  navigation  was  selected  because  the  Naval  Postgraduate  School  has  conducted  a 
series  of  studies  involving  cognitive  modeling  of  group  behaviors  within  real  and  virtual 
navigation  tasks.  Previous  research  had  also  indicated  that  while  conducting  small  group 
navigation  exercises,  the  members  of  the  group  were  very  involved  in  collaborating  when 
they  were  disoriented.  (Boswell,  2000)  This  provided  a  nice  venue  for  developing  a 
navigation  task  that  purposely  placed  the  participants  in  a  disoriented  state.  In  this  way, 
extensive  collaboration  could  be  assured. 

1.  Task  Objective 

The  general  task  objective  was  to  search  for  targets  or  markers  within  the  virtual 
environment,  and  when  located,  plot  those  locations  on  a  map.  During  the  experiment  in¬ 
brief,  each  participant  was  provided  guidance  on  this  objective.  An  explanation  of  the 
collaborative  and  navigational  tools  was  presented  and  the  group  as  a  whole  had  an 
opportunity  to  experience  how  to  navigate  and  communicate  within  a  practice 
environment.  Prior  to  entering  into  the  experimental  environment,  each  participant  was 
again  advised  that  the  goal  of  the  group  was  to  work  together  to  identify  the  location  and 
orientation  of  the  group  as  a  whole  and  to  develop  and  implement  a  strategy  for  locating 
targets.  Each  participant  was  provided  an  outline  map  of  the  environment  for  marking 
purposes.  It  was  left  up  to  the  group  members  to  decide  what  roles,  if  any,  each  member 
would  take  to  solve  the  problem. 
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2.  Navigation  Tools 

In  all  land- navigation  tasks  specific  tools  are  required  to  achieve  the  objective. 
These  tools  provide  the  user  the  ability  to  identify  current  location,  orientation  to  some 
known  reference  point,  and  relative  scale  of  the  environment.  For  this  study,  several 
tools  were  introduced  to  the  users  to  assist  in  achieving  their  objective.  As  a  whole,  the 
group  had  the  following  set  of  tools  available: 

•  Compass 

•  Overhead  color  imagery  map  depicting  vegetation  and  roads 

•  Terrain  contour  map  with  a  compass  rose  identifying  North 

•  Distance  traveled  in  meters  indicator 

•  Photos  of  key  landmarks  within  the  virtual  environment 

•  Map  identifying  locations  of  the  key  landmarks 

If  any  single  person  had  all  these  tools  available  they  would  be  able  to  move  through  the 
environment  with  confidence  and  be  able  to  effectively  identify  their  location.  To  ensure 
that  each  member  of  the  group  was  fully  involved  in  collaborating  about  the  navigation 
task,  each  member  was  supplied  with  only  a  portion  of  the  tools  required  to  effectively 
complete  the  task.  In  this  manner,  each  member  was  required  to  communicate  what  tools 
they  had  available  for  the  task  and  how  they  could  help  solve  the  problem.  The  grouping 
and  distribution  of  the  navigation  tools  is  described  below  in  the  variable  manipulation 
section. 

3.  Group  Coordination 

Given  the  limited  set  of  tools  available  to  each  group  member,  some  level  of 
coordination  was  required.  During  the  in- brief  for  the  experiment,  each  participant  was 
provided  guidance  on  what  tools  were  available  to  the  group  as  a  whole,  but  not  who 
possessed  each  of  the  tools.  Additionally,  the  group  was  instructed  that  any  movements 
and  decisions  would  need  to  be  made  as  a  group.  That  is  to  say,  that  no  one  participant 
was  free  to  roam  around  the  environment  alone.  These  attributes  of  the  task  induced 
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more  collaboration  among  the  participants  on  how  to  complete  the  task.  It  also  increased 
the  likelihood  of  one  participant  emerging  as  a  leader  as  final  decisions  were  established. 

4.  Group  Member  Anonymity 

The  study  required  that  participants  in  any  one  group  not  be  unduly  familiar  with 
each  other.  Lester  (2001)  suggests  this  requirement  must  be  adhered  to  keep  the  level  of 
trust  between  participants  to  a  level  of  “cognitive- based  trust”.  Without  this  limitation 
the  participants  may  have  had  different  interactions  due  to  anticipated  actions  of  others 
caused  by  familiarity.  Every  group  was  developed  through  coordination  by  the  principle 
investigator  of  each  potential  participant’s  schedule.  At  no  time  during  the  recruiting 
process  were  any  of  the  participants  exposed  to  whom  else  may  be  participating  in  the 
experiment.  Each  group  member  was  briefed  individually  prior  to  starting  the 

experiment  task  and  only  told  that  there  were  two  other  members  of  their  group. 

Anonymity  was  also  provided  during  the  group  collaboration  by  assigning  generic 
names  to  each  of  the  group  members.  As  all  the  participants  had  military  experience  and 
they  were  represented  within  the  CVE  as  soldiers,  traditional  military  communication 
callsigns  were  used  for  identification  purposes.  Each  member  was  assigned  the  callsign 
of  “Echo”,  “Kilo”,  or  “Tango”  throughout  the  experiment.  Each  participant  was 
instructed  to  only  use  these  callsigns  for  communicating.  The  callsign  was  also  attached 
to  corresponding  avatar  for  identification  purposes. 

C.  VARIABLE  MANIPULATION 

The  primary  purpose  of  this  study  was  to  ascertain  the  potential  effects  CVE 
interfaces  can  have  on  emergent  leadership.  When  studying  human  factors  related  issues 
there  can  be  some  variables  that  are  difficult  if  not  impossible  to  control.  Some  of  these 
variables  may  not  even  be  recognized  as  a  factor.  The  formation  of  the  groups  as 
described  above  was  a  direct  attempt  to  minimize  any  confounding  factors.  This  should 
allow  for  the  only  independent  variables  to  be  the  interface  used  by  participants  and  the 
tools  used  for  collaboration. 
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1.  Rotating  Interface 

The  interface  was  the  variable  that  was  expected  to  have  the  highest  potential 
impact  on  emergent  leadership  within  the  group.  Each  participant  was  exposed  to  each  of 
the  three  different  levels  of  immersion  into  the  CVE.  If  the  interface  did  have  an  affect 
on  leadership,  as  suggested  by  Slater  (2000),  then  the  most  immersive  interface  should 
produce  more  emergent  leaders  regardless  of  whether  those  people  had  traits  that  predict 
leadership.  The  other  argument  was  to  look  at  how  often  a  predicted  leader  emerged  as 
the  leader  regardless  of  the  interface.  By  rotating  each  participant  through  all  of  the 
interfaces,  both  of  these  arguments  could  be  studied.  Thus,  each  participant  was  exposed 
to  each  of  the  three  interfaces,  or  immersion  levels,  and  evaluated  for  their  role, 
specifically  leadership  role,  while  experiencing  each  interface.  Additionally,  during  the 
rotation  the  anonymity  of  each  group  member  was  maintained. 

2.  Rotating  Tool  Sets 

The  tools  for  navigating  were  divided  into  three  sets  where  any  one  set  did  not 
provide  enough  input  to  the  user  to  accomplish  the  task  on  their  own.  In  this  way,  each 
member  was  required  to  request  assistance  from  the  other  members.  The  three  sets  of 
tools  were  divided  in  this  way: 

•  Compass,  movement  counter,  and  vegetation  map 

•  Compass  rose,  contour  map,  and  photos  of  targets 

•  Key  feature  location  map  and  photos  of  key  navigation  objects 

Consideration  was  given  to  whether  the  tool  set  selection  criteria  was  a  dominant  factor 
in  who  would  emerge  as  the  leader.  To  ensure  that  one  set  of  tools  was  not  more 
advantageous  to  the  task  and  thus  set  one  participant  apart  by  providing  a  performance 
advantage,  the  tools  were  rotated  between  the  different  CVE  interfaces.  This  rotation 
occurred  between  groups  not  between  individuals.  For  example,  one  group  may  have 
tools  sets  1,  2,  and  3  combined  with  interfaces  1,  2,  and  3  respectively.  Each  participant 
in  this  group  was  then  rotated  between  each  of  the  combinations.  The  next  group  would 
have  tool  sets  1,  2,  and  3  combined  with  interfaces  2,  3,  and  1  respectively.  In  this  way, 
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any  correlations  that  exist  between  interface,  tool  set,  or  personality  and  emergent 
leadership  could  be  identified. 

D.  MEASURING  EMERGENT  LEADERSHIP 

There  is  not  always  a  well-defined  method  for  determining  who  the  leader  is 
within  a  group.  Previous  research  indicates  that  verbal  communication  can  be  used  as  an 
indicator  of  leadership  (Kimble  &  McNeese,  1987;  Slater,  et  al.,  2000).  These  studies 
used  different  methods  to  quantitatively  measure  verbalization,  but  each  looked  at  the 
amount  of  talking  done  by  any  individual  and  the  nature  of  the  talking. 

This  study  used  a  similar  method  to  identify  who  within  the  group  was  perceived 
as  the  leader.  Participants  were  required  to  complete  a  survey  after  conducting  the 
navigation  task  using  each  of  the  different  interfaces.  The  survey,  as  represented  in 
Appendix  B,  consisted  of  a  series  of  questions  that  required  them  to  rate  each  person 
within  the  group,  including  themselves,  on  various  attributes  of  the  collaboration  effort. 
Specifically,  they  rated  each  participant  on  talkativeness,  quantity  of  feedback,  quality  of 
feedback,  effectiveness  of  input,  and  overall  involvement.  For  each  rating,  the  rater  was 
instructed  to  place  a  mark  on  a  linear  scale  from  “Low”  to  “High.”  These  group-peer 
evaluations  were  compiled  as  a  subjective  measure  of  who  had  emerged  as  the  leader 
within  the  group  for  that  portion  of  the  experiment.  After  completing  the  interim  survey, 
each  participant  was  rotated  to  the  next  station  to  continue  the  experiment  with  one  of  the 
other  CVE  interfaces. 

After  rotating  through  each  immersion  level  and  completing  a  survey  for  each 
trial,  the  participants  completed  a  second  survey  (Appendix  B)  that  queried  the  user  on 
aspects  of  the  overall  experiment.  This  survey  included  as  the  last  question  a  rating  of 
each  group  member  on  overall  leadership.  All  of  these  ratings  were  quantified  and  used 
during  the  analysis  to  determine  who  had  emerged  as  the  leader  during  each  exercise. 
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V.  ANALYSIS  AND  RESULTS 


A.  EVALUATING  EMERGENT  LEADERSHIP 

Six  groups  of  three  people  completed  the  initial  personality  survey  and  the  CVE 
task.  As  indicated  earlier,  each  of  the  participants  evaluated  the  other  people  in  their 
group,  including  themselves,  on  behaviors  correlated  with  group  leadership.  These 
evaluations  were  subjective  in  nature  and  attempts  were  made  to  develop  objective 
measures  of  emergent  leadership. 

1.  Recording  Group  Participation 

To  help  evaluate  the  group  members  for  emergent  leadership,  the  actions  and 
behaviors  of  each  participant  were  recorded  on  video  and  audio  equipment  at  the  various 
workstations.  It  was  anticipated  that  these  recordings  could  be  reviewed  after  each 
session  to  produce  objective  measures  for  emergent  leadership.  Specifically,  the  intent  of 
the  recordings  was  to  evaluate  each  individual  for  the  following  objective  criteria. 

•  Amount  of  time  individual  spent  talking 

•  Number  of  orders  or  suggestions  given 

•  Number  of  regroup  or  cooperation  statements 

•  Number  of  positive  group  statements  given 

•  Amount  of  time  individual  led  the  group  formation  within  the  CVE 

•  Number  of  final  decision  made  by  individual 

These  criteria  could  then  be  talked  for  each  individual  and  entered  into  a  formula  for 
evaluating  the  individual  as  the  emergent  leader.  Unfortunately,  while  reviewing  these 
recordings,  it  was  determined  that  some  of  the  data  was  garbled  or  it  was  difficult  to 
accurately  identify  which  actions  were  taken  by  which  participants.  It  was  deemed 
inappropriate  to  use  these  recordings  for  creating  objective  measures  due  to  the  inability 
to  determine  which  participant  was  speaking  each  utterance.  However,  the  recordings 
were  useful  for  improving  the  context  of  comments  made  by  the  participants  on  their 
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post- experiment  questionnaires  and  interviews.  This  information  can  be  applied  to  future 
research  in  small  group  collaborations. 

2.  Interim  Questionnaires 

Each  group  collaborated  to  solve  the  navigation  task  and  within  each  group 
different  people  used  different  styles  for  collaboration,  such  as  aggressive,  passive, 
dismptive,  etc..  These  differences  represent  the  heart  of  this  research  effort.  Individual 
personality  traits  are  one  factor  involved  in  the  exhibition  of  these  differences.  Those 
individuals  with  personality  traits  associated  with  emergent  leadership  (extraverted, 
conscientious,  etc.)  should  display  those  traits  in  a  physical  way  (i.e.  talkativeness)  and 
then  emerge  as  the  leader. 

a.  Scoring 

The  completed  interim  questionnaires  were  converted  to  a  database  of 
scores  associated  with  each  participant.  Each  participant’s  mark  for  a  given  measure  was 
represented  as  a  value  from  0  to  10.  The  raw  data  is  represented  in  Appendix  C.  As  each 
group  had  three  members  and  each  survey  had  five  measures,  each  participant  was 
assigned  15  values  for  each  of  the  three  different  CVE  interfaces  utilized. 

3.  Post  Questionnaires 

After  completing  the  task  within  each  of  the  three  CVE  interfaces  and  the 
associated  interim  questionnaires,  the  participants  were  directed  to  complete  a  final 
survey  (Appendix  B)  to  elicit  general  comments  regarding  the  CVE  interfaces  and  group 
collaboration.  An  additional  measure  was  taken  for  each  group  member  to  evaluate  the 
others  as  to  their  role  as  the  overall  leader  of  the  group.  This  evaluation  was  converted  to 
a  numeric  value  in  the  same  fashion  as  the  questions  on  the  interim  survey. 

B.  ANALYSIS  OF  MEASURED  LEADERSHIP  SCORES 

Two  main  avenues  of  approach  were  taken  for  analyzing  the  data  collected  from 

the  18  participants.  The  first  approach  was  to  look  at  the  scores  in  a  general  way  similar 

to  the  way  Slater  (2000)  measured  the  emergent  leader  in  his  research.  The  second 

approach  entailed  a  statistical  correlation  of  personality,  interface,  and  emergent 
leadership. 
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1.  General  Analysis 

In  the  research  conducted  by  Slater  emergent  leadership  was  not  the  primary 
focus.  It  was  merely  an  observation  of  interest  that  bore  notation.  The  approach  used 
was  to  evaluate  the  mean  and  standard  deviations  for  the  leadership  scores  of  each 
individual  related  to  the  interface  they  used  within  the  environment.  He  also  analyzed  the 
number  of  times  a  particular  individual  had  the  highest  score  amongst  the  other  members 
of  the  same  group. 

Taking  this  approach,  this  study  summed  and  averaged  each  of  the  values  from 
the  interim  questionnaires  for  each  participant.  The  range  of  summed  values  for  each 
measurement  on  the  questionnaire  was  0  -  30,  as  there  were  three  people  evaluating  that 
characteristic.  The  values  for  each  characteristic  were  also  added  to  create  a  range  of 
possible  values  from  0-150  for  any  one  participant  at  a  particular  CVE  interface.  These 
scores  were  then  tabulated  according  to  personality  traits  and  interfaces.  Specifically,  for 
each  individual  they  were  categorized  as  “predicted  leaders”  or  “non- leaders”  according 
to  personality  traits  and  those  scores  across  all  interfaces  were  summed  and  average. 
Additionally,  the  scores  across  all  the  individuals  at  each  of  the  three  different  immersion 
workstations  were  also  categorized.  The  number  of  times  a  particular  individual’s  scores, 
or  the  scores  of  the  people  using  a  particular  interface,  was  the  highest  was  also  recorded. 
Tables  3  and  4  show  the  relationship  of  these  scores.  Some  information  can  be  gained 
from  these  tables,  but  only  in  an  exploratory  way. 


Total  Score 

Avg 

Std  Dev 

Frequency 

Leader 

1369.5 

273.9 

26.5 

43.0 

Non-leader  1 

1010.4 

202.1 

24.1 

21.0 

Non-leader  2 

1203.6 

240.7 

23.7 

26.0 

Table  3.  Mean,  Standard  Deviation  and  Frequency  of  Highest  Score  per  Personality. 
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Total  Score 

Avg 

Std  Dev 

Frequency 

Cave 

1565.4 

313.1 

20.1 

30.5 

PV230 

1507.1 

301.4 

23.4 

24.5 

Single 

1480.4 

296.1 

19.3 

35.0  I 

Table  4.  Mean,  Standard  Deviation  and  Frequency  of  Fiighest  Score  per  Interface. 

2.  Statistical  Analysis 

For  a  more  complete  analysis,  scores  for  each  participant  and  each  immersion 
interface  were  analyzed  using  analysis  of  variance  (ANOVA).  To  analyze  the  effects  the 
interface  had  on  potential  emergent  leadership,  a  1-way  ANOVA  was  conducted  with 
three  treatments.  The  box  plot  of  the  data  (Figure  6)  shows  the  distribution  of  the  scores 
for  each  interface.  The  results  of  the  statistical  ANOVA  show  an  acceptance  of  the  null 
hypothesis  that  all  the  means  are  equal  with  a  probability  of  0.8  (F-test:  2,  51  degrees  of 
freedom).  A  post-hoc  power  analysis  was  conducted  to  recognize  possible  Type  II  errors 
-  acceptance  of  null  hypothesis  that  was  actually  false.  Using  an  a  =  0.1  on  3  treatments 
with  a  population  of  18  a  low  power  (1  -  (3)  =  0.39  was  calculated.  This  indicates  there  is 
a  high  probability  that  a  significant  difference  between  treatments  did  exist  but  could  not 
be  detected  due  to  low  sample  size. 
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Figure  6.  Distribution  of  Emergent  Leadership  Scores  by  Interface. 


A  slight  variation  in  approach  was  used  for  analyzing  the  effect  personality  traits 
have  on  emergent  leadership.  Although  each  participant  had  different  personality  traits, 
there  was  only  a  distinction  made  between  the  individual  that  had  the  traits  related  to 
emergent  leadership  and  those  that  did  not  have  those  traits.  This  distinction  required  an 
analysis  between  two  groups,  leaders  and  non-leaders,  with  different  population  counts. 
The  paired  Ttest  was  inappropriate  for  this  dataset,  so  a  1-way  ANOVA  was  again  used 
to  account  for  the  different  populations.  Figure  7  shows  the  distribution  of  the  scores  for 
the  two  personality  categories  across  all  interfaces.  The  result  of  this  analysis  was  to 
reject  the  null  hypothesis  that  the  means  were  equal.  The  Ftest  yields  /  =  6.84  is  greater 
than  F(o.o5,  t,  52)  =  4.02  (p  =  0.012). 
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Measured  Leadership  Scores  For  Predicted  Leaders 


Personality 


Figure  7.  Distribution  of  Emergent  Leadership  Scores  by  Personality  Across  All  Interfaces. 

One  additional  analysis  was  conducted,  but  did  not  produce  significant  results. 
Using  the  data  gained  from  the  post  questionnaire,  which  required  the  participants  to  rate 
each  other  as  overall  leader,  another  unbalanced  ANOVA  was  completed.  The  box  plot 
in  Figure  8  shows  the  distribution  of  data.  The  results  of  this  analysis  were  to  not  reject 
the  null  hypothesis  that  the  means  were  equal.  The  Ftest  yields  /  =  4.37  is  not  greater 
than  F(0.o5,  i,  16)  =  4.49  ( p  =  0.053). 
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Figure  8.  Distribution  of  Overall  Leadership  Scores  by  Personality. 

C.  ANALYSIS  DISCUSSION 

The  results  obtained  from  looking  at  the  mean  and  frequency  of  selection  provides 

some  insight  into  the  group  collaboration.  Table  3  clearly  shows  that  those  individuals 

possessing  the  “emergent  leader”  traits  scored  higher,  on  average,  than  the  other  group 

members.  It  is  even  more  noticeable  when  examining  the  frequency  for  people  who 

received  the  highest  leadership  characteristics  scores.  In  these  cases,  the  predicted 

leaders  were  selected  nearly  2  to  1  over  the  other  group  members.  This  result  was 

confirmed  through  statistical  analysis  using  the  unbalanced  ANOVA  with  an  almost  99 

percent  confidence  interval.  Additionally,  results  obtained  from  evaluation  of  each  group 

member  as  the  “overall  leader”  reveal  similar  results.  The  analysis  did  not  produce  a 

significant  result,  but  this  may  be  due  to  the  small  population  size.  Those  results  show 

significance  at  an  alpha  level  of  0.10,  but  not  at  0.05. 
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The  results  regarding  interface  effects  differs  from  the  results  obtained  in  Slater’s 
research.  The  study  reported  here  found  some  variation  between  mean  leadership  scores 
and  the  number  of  times  these  participants  were  selected  as  leaders,  but  does  not 
demonstrate  any  clear  relationship  between  these  two  variables.  The  results  obtained 
from  the  ANOVA  indicate  the  mean  leadership  scores  were  equal  between  the  different 
interfaces. 

There  are  several  explanations  to  account  for  the  differences  observed.  During 
Slater’s  (2000)  experiment  all  the  group  members  were  required  to  collaborate  on  the 
task,  but  one  of  the  participants  was  instructed  to  nonchalantly  obstruct  the  view  of  one  of 
the  other  participants.  This  additional  duty  was  also  never  given  to  the  participant  with 
the  most  immersive  interface  (i.e.  Head  Mounted  Device).  As  this  additional  duty  could 
potentially  detract  from  the  ability  of  the  person  to  communicate  and  collaborate,  then 
they  would  be  at  a  disadvantage.  This  study  did  not  implement  any  additional  tasks  in 
the  manner  of  Slater’s  study.  So,  this  could  account  for  some  of  the  difference  but  not 
all.  Another  consideration  is  the  use  of  the  HMD.  In  general,  a  HMD  will  provide  a 
more  immersive  interface  than  the  CAVE- like  interface  used  in  this  study.  The  physical 
field  of  view  and  view  orientation  attributes  of  the  HMD  also  produce  different 
interactions  with  the  CVE  and  the  real  world.  The  HMD  does  not  easily  allow  the  user  to 
switch  between  real  and  virtual  worlds.  Hence,  the  user  tends  to  become  more  immersed 
over  time.  This  study  required  the  users  to  review  maps  that  necessitated  switching 
between  real  and  virtual  environments  to  complete  the  task.  This  switching  could  have 
reduced  the  ability  of  the  various  interfaces  to  fully  the  immerse  the  participants  in  the 
virtual  environment.  An  additional  note,  the  participants  in  Slater’s  study  were  involved 
in  performing  the  task  in  the  CVE  for  a  greater  time  period  than  this  study  which  also 
may  have  had  an  effect  on  the  degree  of  immersion. 

Hie  conclusions  to  be  gained  from  these  analyses  are  that  the  interface  used  with 

a  CVE,  or  the  level  of  immersion  the  interface  provides  to  the  user,  does  not  appear  to 

significantly  affect  the  emergence  of  leadership.  Further  study  must  be  given  to  this  area 

of  collaborative  virtual  environments  in  order  to  explain  the  differences  found  between 

this  research  and  Slater’s.  A  larger  sample  size  might  have  permitted  forming  groups 

based  on  more  extreme  scores  which  might  have  produced  the  anticipated  results. 
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Additionally,  the  personality  traits  associated  with  emergent  leadership  do  play  a 
significant  role  in  emergent  leadership  within  collaborative  virtual  environments. 
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V.  CONCLUSION  AND  FUTURE  WORK 


A.  RESTATEMENT  OF  OBJECTIVE 

As  technologies  like  high-speed  networks,  audio  and  video  conferencing,  and 
virtual  reality  improve  and  gain  more  acceptance,  people  will  be  more  involved  in 
collaborative  work  between  distant  locations.  Collaborative  virtual  environments  may 
play  a  significant  role  and  we  must  better  understand  the  implications  involved  in  their 
use.  This  is  especially  important  when  these  systems  are  used  for  training  to  help  ensure 
adverse  training  effects  are  avoided. 

This  study  was  designed  to  whittle  away  at  some  of  the  unknown  quantities  of 
collaboration  within  distributed  virtual  environments.  To  this  end,  the  study  looked 
specifically  at  the  relationship  between  system  interface,  personality  traits,  and  emergent 
leadership  within  a  collaborative  virtual  environment.  By  using  the  predictability 
personality  traits  provide  for  detecting  emergent  leadership,  this  research  evaluated  who 
emerged  as  the  leader  during  small  group  collaboration  and  compared  this  with  who  was 
the  predicted  leader  based  on  personality  traits.  As  each  group  member  was  exposed  to 
three  different  interfaces  each  with  a  different  level  of  immersion,  any  effects  the  varied 
interface  provided  to  the  users  should  have  been  uncovered.  It  was  theorized,  based  on 
previous  research  (Slater,  et  al.,  2000),  that  the  limitations  of  low-end  interfaces  would 
negatively  impact  the  transfer  of  leadership  personality  traits  within  the  virtual 
environment,  but  not  to  such  a  degree  that  the  limitations  could  not  be  overcome. 

B.  CONCLUSIONS 

Two  main  conclusions  were  formed  based  on  the  results  of  the  experiment.  The 
first  of  these  conclusions  is  directly  related  to  emergent  leadership  personality  traits  and 
the  applicability  of  these  traits  within  virtual  environments.  The  conclusion  states: 

•  The  personality  traits  associated  with  emergent  leadership  do  play  a 
significant  role  in  emergent  leadership  within  collaborative  virtual 
environments. 


The  second  conclusion  actually  argues  against  the  conclusions  generated  in  previous 

research  by  Slater  (2000)  at  University  College,  London.  Slater’s  results  indicated  that 

37 


the  interface  is  a  major  contributing  factor  to  emergent  leadership,  but  the  results  of  this 
research  conclude: 

•  The  interface  to  a  CVE,  or  the  level  of  immersion  the  interface  presents  to 
the  user,  may  not  significantly  affect  the  emergence  of  leadership. 

There  may  be  several  possible  explanations  for  the  difference  in  conclusions,  and  one 
significant  point  is  that  Slater’s  research  did  not  take  the  dynamics  of  personality  into 
account  while  studying  the  group  interaction. 

C.  FUTURE  WORK 

One  point  is  clear  when  looking  at  the  results  of  this  research,  Slater’s,  and  others 
like  it,  is  that  further  research  is  needed  to  fully  understand  the  effects  virtual 
environment  technologies  have  on  group  collaboration.  Specific  attention  must  be  paid 
to  groups  that  collaborate  within  a  shared  virtual  environment  when  the  participants  are 
not  co- located. 

Many  factors  are  involved  in  the  dynamics  of  group  interactions  in  addition  to 
personality.  This  research  took  precautions  to  ensure  the  participants  within  each  group 
remained  anonymous  and  were  gender  homogenous.  Other  research  indicates  that 
attributes,  whether  simple  or  complex,  can  have  an  impact  on  group  dynamics  and  gender 
is  among  these  attributes  (Kimble  &  McNeese,  1987).  Group  dynamics  become  more 
complicated  when  several  factors  are  considered:  group  gender  mix,  leader  gender,  etc. 
Future  research  should  consider  gender  and  other  attributes  of  group  dynamics  and  how 
different  CVE  interfaces  affect  the  roles  people  take  within  groups.  Below  is  a  partial  list 
of  topics  for  future  work. 

•  Emergent  leadership  in  groups  with  a  gender  mix 

•  Measuring  leadership  qualities  in  groups  with  appointed  leaders 

•  Effects  different  interfaces  have  on  training  leadership  within  small  groups 

•  Evaluating  effects  CVE  interfaces  have  on  things  other  than  leadership 
(aggression,  stress  resolution,  situational  awareness,  etc.) 
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•  CVE  interface  effects  within  domains  other  than  land  navigation  or 
reconnaissance  (personnel  management,  academic  research,  training,  etc.) 

•  Comparison  between  real  and  virtual  world  group  dynamics  for  similar 
tasks 

•  Qualities  required  of  a  CVE  to  support  effective  collaboration 

Hie  United  States  military  is  deeply  involved  in  researching  the  use  of  virtual 
environment  technologies  for  training  individuals  and  teams.  Virtual  environments  are  a 
logical  choice  for  some  training  platforms  due  to  the  costs  and  dangers  involved  in  some 
of  the  training  requirements.  Before  these  training  systems  are  implemented  for  field  use, 
a  thorough  understanding  of  the  effects  different  CVE  interfaces  provide  must  be 
achieved.  With  a  good  understanding  of  these  effects  the  best  systems  can  be  employed 
to  produce  positive  training  transfer  for  individuals  and  groups. 
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APPENDIX  A.  VIRTUAL  MODEL  IMPLEMENTATION 


A.  HARDWARE  CONFIGURATION 

This  appendix  provides  the  detailed  information  on  how  the  hardware  and 
software  was  configured  to  support  this  research. 

1.  High-Immersion  Workstation 

Duiing  the  planning  stage  of  this  research  special  consideration  was  given  to  the 
hardware  requirements  for  this  workstation.  The  research  conducted  by  Slater  utilized  a 
Head-Mounted  Display  (HMD)  for  the  high- immersion  workstation.  For  similarity 
purposes,  a  HMD  type  system  was  desired,  but  the  collaborative  task  made  using  a  HMD 
impractical.  The  task  required  the  user  to  interact  with  “real-world”  tools  like  maps  in 
order  to  achieve  the  objectives.  The  resolution  capabilities  of  current  HMD  technologies 
are  not  sufficient  to  support  detailed  map  displays.  Thus,  a  CAVE- like  system  was 
utilized  to  provide  a  highly  immersive  interface  to  the  user. 
a.  Display  Hardware 

The  display  system  utilized  a  Multi- Angle  Virtual  Environment  (MAVE) 
developed  from  previous  research  at  NPS  (Figure  9).  The  MAVE  consisted  of  three  large 
polarization- preserving  rear-projection  screens  that  were  each  7  feet  wide  and  6  feet  high. 
Each  of  these  screens  was  raised  above  the  floor  22  inches  to  provide  a  more  natural 
viewing  angle  to  a  participant  in  a  standing  posture.  For  this  experiment  the  participants 
were  provided  an  elevated  stool  to  reduce  fatigue  and  facilitate  an  eye  position 
approximating  a  standing  posture.  The  three  displays  were  placed  at  45  degree  angles  to 
each  other  and  produced  a  physical  field  of  view  of  approximately  135  degrees  when  the 
user  was  at  the  optimal  viewing  distance  from  the  center  display  of  approximately  6  feet. 
Each  display  had  a  dedicated  VRex  VR2210  polarized  stereo  LCD  projector  that 
projected  the  image  onto  the  screens  via  two  reflecting  mirrors.  The  three  displays 
existed  within  a  small  room  with  sound  absorbent  surfaces  and  no  external  distraction. 
Overall  this  provided  a  highly  immersive  environment  for  the  experiment  participants. 
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Figure  9.  User  at  High- Immersion  Workstation. 
b.  Computer  Hardware 

This  workstation  required  the  highest  level  of  computational  power.  This 
was  achieved  by  building  three  machines  dedicated  to  rendering  the  CVE  images  to  the 
three  projectors.  Each  machine  was  produced  from  commercial  off-the-shelf  products  as 
indicated  in  Table  1.  The  graphical  rendering  synchronization  between  the  machines  was 
controlled  by  software  over  a  dedicated  high-speed  1000  Megahertz.  Overall,  the 
computer  configuration  smoothly  reproduced  the  CVE  with  average  frame  rates  greater 
than  25  frames  per  second  at  a  resolution  of  1024  by  768  pixels. 

2.  Mid-Immersion  Workstation 

The  basic  requirements  for  this  workstation  were  to  provide  the  user  an  interface 
to  Ihe  CVE  that  was  more  immersive  than  a  single- screen  display  and  less  than  that  of  the 
high- immersion  workstation. 

a.  Display  Hardware 

The  display  system  utilized  a  specialized  panoramic  display  (PV230) 
developed  by  Panoram  Technologies  (Figure  10).  The  display  consisted  of  three  mid¬ 
sized  flat  panel  LCD  screens  connected  together  at  approximately  25  degree  angles.  The 
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seams  between  each  display  were  approximately  1 1  millimeters  and  produced  little 
distraction  when  the  user’s  focal  point  transitioned  between  the  displays.  Each  individual 
display  had  a  15  inch  diagonal  measurement  and  all  combined  provided  a  36  inch  wide 
by  9  inch  high  viewing  area.  Overall  this  provided  a  pleasing,  high  quality  display  with  a 
wide  field  of  view,  but  not  as  immersive  a  display  as  the  high- immersion  workstation. 


Figure  10.  User  at  Mid-Immersion  Workstation. 
b.  Computer  Hardware 

This  workstation  utilized  a  high-end  Dell  personal  computer  augmented  to 
support  multiple  displays  as  indicated  in  Table  2.  Due  to  the  complexities  of  the  virtual 
model,  the  computer  configuration  reproduced  the  CVE  with  a  disappointing  average 
frame  rate  of  approximately  10  frames  per  second  at  a  resolution  of  1024  by  768  pixels. 
Although  this  frame  rate  could  have  been  improved  by  reducing  the  image  resolution,  it 
was  considered  more  important  to  ensure  each  workstation  had  identical  pixel 
resolutions.  The  frame  rate  difference  was  noticeable,  but  participant  post- experiment 
comments  regarding  frame  rate  did  not  reveal  any  significant  display  problems.  It  is 
theorized  the  slow  movement  associated  with  “walking”  in  the  CVE  allowed  the  users  to 
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cope  with  the  low  frame  rate.  The  difference  in  frame  rate  was  most  noticeable  while 
changing  view  azimuth  or  elevation. 

3.  Low-Immersion  Workstation 

The  basic  requirement  for  this  workstation  was  to  provide  the  user  an  interface  to 
the  CVE  that  had  few  immersive  qualities,  as  found  in  typical  personal  computer 
systems. 

a.  Display  Hardware 

The  display  system  utilized  a  single  21  inch  diagonal  monitor  as  part  of  a 
multiple  display  system;  the  additional  displays  were  not  utilized  for  the  experiment. 
Although  the  diagonal  measurement  of  this  display  was  greater  than  the  mid- immersion 
workstation,  the  total  viewing  area  was  less  and  covered  a  much  smaller  physical  field  of 
view.  Overall,  this  interface  provided  the  least  capability  to  immerse  the  participant  in 
the  CVE. 


Figure  11.  User  at  Low-  Immersion  Workstation. 
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b.  Computer  Hardware 

This  workstation  utilized  a  high-end  Intergraph  TDZ  2000  personal 
computer  augmented  to  support  multiple  displays  as  indicated  in  Table  2.  Overall,  the 
computer  configuration  reproduced  the  CVE  with  an  average  frame  rate  of  approximately 
20  frames  per  second  at  a  resolution  of  1024  by  768  pixels. 

B.  SOFTWARE  CONFIGURATION 

All  the  CVE  workstations  utilized  the  same  software  with  minor  variations  to 
support  the  different  interface  requirements.  The  real-time  simulation  software  utilized 
for  the  CVE  was  Multigen- Paradigm’s  Vega  software.  Several  modules  were  attached  to 
the  software  kernel  to  support  the  requirements  of  the  experiment.  The  list  below 
identifies  the  main  modules  of  concern  and  their  use  requirement. 

•  ‘  ‘DI-  Guy'  ’  for  avatar  implementation 

•  “DIS/HLA"  for  network  protocol  support 

•  “Symbology"  for  implementation  of  a  compass  and  distance  display 

•  “Distributed  Vega”  for  multi- machine  multi- pipe  scene  rendering 

All  the  modules  were  available  at  every  workstation,  but  the  symbology  and  distributed 
Vega  modules  were  only  activated  on  the  computer  systems  that  required  that 
implementation.  Specifically,  the  Distributed  Vega  module  was  only  activated  for  the 
high- immersion  workstation  and  Symbology  was  only  implemented  for  users  that  had  the 
compass  as  part  of  their  tool  set. 

One  complication  that  occurred  during  this  experiment  was  that  not  all  the 
modules  functioned  together  properly.  An  exorbitant  amount  of  time  was  spent 
configuring  the  software  to  adequately  support  effective  user  control  and  display  of  an 
avatar  within  the  networked  virtual  environment.  The  DIS  module  makes  sharing 
environments  and  virtual  objects  across  networks  very  simple  to  implement. 
Additionally,  the  DI-Guy  module  is  a  great  way  to  introduce  humanoid  objects  into  a 
CVE,  but  they  were  designed  for  scripted  behaviors.  Another  complication  was  that  the 
DI-Guy  is  represented  within  the  Vega  software  as  a  special  kind  of  object.  As  a  special 
object,  the  avatar  is  not  sent  across  networks  by  the  DIS  module  in  the  same  way  as  other 
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objects.  In  essence,  these  peculiarities  of  the  software  complicated  the  development 
process.  The  next  sections  document  some  of  the  solutions  implemented  to  support  this 
research.  Specifically,  the  code  implementation  to  allow  a  user  to  control  an  avatar  and 
then  represent  the  state  of  that  avatar  at  the  receiving  workstations  will  be  described. 

1.  Avatar  Control 

The  DI-Guy  software  module  was  developed  to  support  the  introduction  of 
humanoid  objects  with  realistic  human  behaviors  into  real-time  simulations.  These 
objectives  have  been  met  quite  effectively,  but  are  done  using  pre- scripted  behaviors  and 
movement  paths.  There  are  hooks  available  to  allow  software  to  dynamically  control  the 
actions  of  the  avatar,  but  when  these  are  utilized  the  realism  of  the  behaviors  and  actions 
is  reduced. 

The  requirements  of  this  study  included  the  ability  of  the  user  to  point  at  objects 
or  in  a  given  direction.  To  support  this,  the  avatar  required  direct  user  control  and  all  the 
group  members  needed  to  see  the  pointing  behavior.  Control  was  achieved  through 
keyboard  actions  that  translated  to  a  change  of  avatar  “state”.  The  code  fragment  below 
(Figure  12)  illustrates  how  those  keyboard  commands  were  utilized.  When  a  keyboard 
event  was  captured  the  event  information  was  parsed  to  determine  which  key  was  pressed 
and  then  the  appropriate  avatar  state  was  set  and  that  state  was  transmitted  over  the 
network  to  the  workstations.  An  additional  adjustment  was  also  made  to  the  viewing 
height  of  the  user  dependent  on  whether  the  avatar  state  changed  from  standing  to 
kneeling. 
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switch (keyPressed)  { 

//*******  the  following  key  events  are  for  DI-Guy  motions  ****** 
case  VGWIN_UPARROW :  //character  walk  (property  value  =  6) 
case  VGWIN_PAD 8 : 

//******  change  avatar  state 

vgDIGuyAction  ( vgCharHandle ,  SOLDIER_WALK) ; 

//******  adjust  observer  view  point 
chgObsOf  f  set Alt (SOLDIER_WALK)  ; 

//******  send  State  change  info  to  DIS 

sendState ( SOLD IER_WALK) ; 

break; 

case  VGWIN_PAD9:  //character  jog  (property  value  =  11) 
case  ' r '  : 
case  ' R ' : 

//******  change  avatar  state 
vgDIGuyAction  (vgCharHandle,  SOLDIER_JOG) ; 

//******  adjust  observer  view  point 
chgObsOf  f  set Alt (SOLDIER_JOG)  ; 

//******  send  State  change  info  to  DIS 

sendState (SOLDIER_JOG) ; 

break; 


Figure  12.  Code  to  Control  DI-Guy  Avatar  State. 


The  pre- scripted  behaviors  for  which  the  DI-Guy  object  was  designed  utilized 
transitions  between  states  to  achieve  the  realistic  human  actions  when  switching  avatar 
states.  For  instance,  if  the  avatar  switched  from  running  to  standing,  then  the  script 
would  actually  adjust  the  avatar  from  a  running  state  to  a  walking  state  and  then  a 
standing  state.  This  created  a  visually  realistic  state  transition.  When  sending  state 
information  about  the  avatar  over  the  network  these  intermediate  states  must  be  sent  as 
well.  For  simplicity  reasons,  these  intermediate  states  were  not  transmitted  which  could 
cause  the  avatar  on  the  receiving  workstation  to  get  stuck  in  a  particular  state.  To  remedy 
this  feature,  the  state  transitions  were  disabled  and  the  avatar  immediately  changed  state. 
This  created  some  jerky  actions  when  transitioning  between  certain  avatar  states. 

2.  Sending  and  Receiving  Avatar  State  Information 

The  DIS  network  module  was  designed  to  automatically  send  state  and  positional 
information  about  objects  to  all  the  nodes  in  the  shared  CVE.  Unfortunately,  the  DI-Guy 
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is  treated  as  a  special  type  of  object  and  this  information  is  not  captured  and  sent  properly 
when  the  avatar  is  under  user  control.  Thus,  the  state  information  of  the  avatar  was 
manually  sent  to  the  other  nodes  on  the  network.  This  was  implemented  via  a  separate 
multicast  channel  established  between  all  the  workstations.  Sending  the  information  was 
simple  and  straightforward;  the  complications  occurred  on  the  receive  side. 

Each  workstation  had  an  established  identification  number  within  the  CVE  and 
this  number  attached  to  the  avatar  state  information  that  was  sent  to  the  other 
workstations.  Upon  receipt,  the  receiving  node  would  parse  the  network  packet  for  the 
station  identification  and  then  locate  the  associated  avatar  that  needed  to  be  adjusted. 
Typically,  this  locating  of  the  associated  avatar  would  be  a  trivial  search  for  the  object 
name  and  assigning  a  pointer,  but  because  of  the  special  nature  of  the  DI-Guy  objects  this 
was  complicated.  The  DI-Guy  objects  that  represent  another  workstation  user  are 
dynamically  created  and  destroyed  during  the  simulation,  and  each  time  this  happens  a 
new  name  is  assigned  to  the  object.  Thus,  the  name  of  the  object  is  never  known.  To  get 
around  this  problem,  each  user’s  avatar  was  assigned  a  specific  name  and  that  name  was 
linked  to  the  workstation  identification  number.  Then,  the  receiving  workstation  could 
identify  each  avatar  by  the  link  between  the  name  and  the  number. 

3.  Adjusting  Avatar  Position 

Another  compatibility  problem  between  the  modules  appeared  in  relation  to  the 
position  of  the  avatar  on  the  receiving  station.  The  DIS  module  automatically  reported 
the  position  information  of  an  object  within  the  CVE.  For  DI-Guy  objects,  the  module 
does  not  send  the  position  information  of  the  avatar,  but  sends  the  position  of  another 
object  that  represents  the  avatar.  When  the  positional  information  is  received  at  a 
workstation  the  DIS  module  automatically  decodes  the  information  and  sets  the  position 
of  another  object  that  represents  the  avatar  rather  than  the  avatar  itself.  This 
representative  object  exists  because  of  the  special  nature  of  DI-Guy.  The  action  states 
and  behaviors  associated  with  the  DI-Guy  avatar  fend  to  make  the  avatar  position  on  the 
receiving  station  shift  in  relation  to  the  representative  object  position.  To  fix  this 
problem  the  avatar  object  position  needs  to  be  periodically  adjusted  back  to  the  position 
of  the  representative  object.  Figure  B  shows  the  function  used  to  make  this  adjustment. 
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This  drift  and  adjustment  occur  for  every  instance  of  networked  DI-Guys,  so  the  function 
simply  adjusts  all  the  inbound  avatar  positions. 


void  diGuyCorrection  ( )  { 

vgDIGuy  *d  =  NULL; 
float  x=0.0f,  y=0.0f,  z=0.0f; 

bdiCharacterHandle  bdiChar  =  NULL; 

//loop  through  all  the  DIS  DI-Guys  and 
//find  the  active  DIS  DIGuy  entities 

for  (int  index  =  0;  index  <  MAX_DIS_DIGUYS;  index++)  { 
//check  if  array  holder  has  a  valid  DIS  entity 
if  (disDIGuyNames [ index]  !=  NULL)  { 

//try  to  find  the  associated  DIGuy 

if  (  (d  =  vgFindDIGuy (disDIGuyNames [index] ) )  !=  NULL)  ( 

//get  the  associated  BDI  Handle 
bdiChar  =  vgGetDIGuyCharacterHandle (d) ; 

//teleport  to  0,0,0 

diguy_teleport (bdiChar ,  &x,  &y,  &z); 

}//end  if 
}//end  if 
}//end  for  loop 
(//end  diGuyCorrection 


Figure  13.  Code  to  Adjust  DI-Guy  Avatar  Position. 


There  were  many  adjustments  made  to  the  experiment  model  to  create 
controllable  avatars  that  could  be  used  by  the  participants  in  a  way  that  was  conducive  to 
the  navigation  task.  Although  not  all  the  problems  were  fixed,  the  participants  did  not 
report  an  inability  to  achieve  the  task  objectives  based  on  avatar  control  and 
representation.  Most  users  enjoyed  the  experience  and  found  the  pointing,  along  with 
other  features,  important  to  achieving  the  objectives. 
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APPENDIX  B.  EXPERIMENT  FORMS 


A.  GENERAL 

Hie  forms  in  this  appendix  appear  in  the  same  format  utilized  for  the  experiment 
and  do  not  follow  the  standard  thesis  formats  utilized  in  the  chapters  of  this  document. 
The  appendix  consists  of  six  documents:  Privacy  Act  Statement,  Participant  Consent 
Form,  Minimal  Risk  Consent  Statement,  In-brief  Statement,  Interim  Questionnaire,  and 
the  Post  Questionnaire. 

1.  Privacy  Act  Statement 

The  following  page  represents  the  Privacy  Act  Statement  presented  to  each 
participant.  Each  participant  was  required  to  read  and  sign  this  document  prior  to 
participating  in  the  experiment. 
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PRIVACY  ACT  STATMENT 


NAVAL  POSTGRADUATE  SCHOOL,  MONTEREY,  CA  93943 
PRIVACY  ACT  STATEMENT 

1.  Authority:  Naval  Instruction 

2.  Purpose:  Personality  information  will  be  collected  to  enhance  knowledge,  and  to 

develop  tests,  procedures,  and  equipment  to  improve  the  development  of  Distributed 
Collaborative  Virtual  Environments. 

3.  Use:  Personality  information  will  be  used  for  statistical  analysis  by  the  Departments 
of  the  Navy  and  Defense,  and  other  U.S.  Government  agencies,  provided  this  use  is 
compatible  with  the  purpose  for  which  the  information  was  collected.  Use  of  the 
information  may  be  granted  to  legitimate  non- government  agencies  or  individuals  by 
the  Naval  Postgraduate  School  in  accordance  with  the  provisions  of  the  Freedom  of 
Information  Act. 

4.  Disclosure/Confidentiality: 

a.  I  have  been  assured  that  my  privacy  will  be  safeguarded.  I  will  be  assigned  a 
control  or  code  number,  which  thereafter  will  be  the  only  identifying  entry  on 
any  of  the  research  records.  The  Principal  Investigator  will  maintain  the  cross- 
reference  between  name  and  control  number.  It  will  be  decoded  only  when 
beneficial  to  me  or  if  some  circumstances,  which  is  not  apparent  at  this  time, 
would  make  it  clear  that  decoding  would  enhance  the  value  of  the  research  data. 
In  all  cases,  the  provisions  of  the  Privacy  Act  Statement  will  be  honored. 

b.  I  understand  that  a  record  of  the  information  contained  in  this  Consent  Statement 
or  derived  from  the  experiment  described  herein  will  be  retained  permanently  at 
the  Naval  Postgraduate  School  or  by  higher  authority.  I  voluntarily  agree  to  its 
disclosure  to  agencies  or  individuals  indicated  in  paragraph  3  and  I  have  been 
informed  that  failure  to  agree  to  such  disclosure  may  negate  the  purpose  for 
which  the  experiment  was  conducted. 

c.  I  also  understand  that  disclosure  of  the  requested  information,  including  my 
Social  Security  Number,  is  voluntary. 


Signature  of  Volunteer  Name,  Grade/Rank  (if  applicable)  DOB  SSN  Date 


Signature  of  Principal  Investigator  Date 
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2.  Participant  Consent  Form 

Hie  following  page  represents  the  Participant  Consent  Form  presented  to  each 
participant.  Each  participant  was  required  to  read  and  sign  this  document  prior  to 
participating  in  the  experiment. 
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PARTICIPANT  CONSENT  FORM 


1.  Introduction.  You  are  invited  to  participate  in  a  study  of  group  interactions  within 
virtual  environments.  With  information  gathered  from  you  and  other  participants,  we 
hope  to  discover  insight  on  communication  techniques  used  while  collaborating  within 
a  virtual  environment  during  dismounted  navigation  of  natural  terrain.  We  ask  you  to 
read  and  sign  this  form  indicating  that  you  agree  to  be  in  the  study.  Please  ask  any 
questions  you  may  have  before  signing. 

2.  Background  Information.  The  Naval  Postgraduate  School  NPSNET  Research  Group 
is  conducting  this  study. 

3.  Proceckires.  If  you  agree  to  participate  in  this  study,  the  researcher  will  explain  the 
tasks  in  detail.  There  will  be  two  sessions:  a)  30  minute  pretest  phase  and  b)  60  minute 
training  and  execution  phases,  during  which  you  will  be  expected  to  accomplish  a 
number  of  tasks  related  to  navigating  a  natural  terrain  environment. 

4.  Risks  and  Benefits.  This  research  involves  no  risks  or  discomforts  greater  then  those 
encountered  in  ordinary  computer  usage.  The  benefits  to  the  participants  are  gaining 
techniques  for  enhancing  spatial  knowledge  of  unfamiliar  environments  and 
contributing  to  current  research  in  human- computer  interaction. 

5.  Compensation.  No  tangible  reward  will  be  given.  A  copy  of  the  results  will  be 
available  to  you  at  the  conclusion  of  the  experiment. 

6.  Confidentiality.  The  records  of  this  study  will  be  kept  confidential.  No  information 
will  be  publicly  accessible  which  could  identify  you  as  a  participant. 

7.  Voluntary  Nature  of  the  Study.  If  you  agree  to  participate,  you  are  free  to  withdraw 
from  the  study  at  any  time  without  prejudice.  You  will  be  provided  a  copy  of  this  form 
for  your  records. 

8.  Points  of  Contact.  If  you  have  any  further  questions  or  comments  after  the  completion 
of  the  study,  you  may  contact  the  research  supervisor,  Dr.  Rudolph  P.  Darken  (831) 
656-4072,  darken@nps.navy.mil. 

9.  Statement  of  Consent.  I  have  read  the  above  information.  I  have  asked  all  questions 
and  have  had  my  questions  answered.  I  agree  to  participate  in  this  study. 


Signature  of  Volunteer 


Date 


Signature  of  Principal  Investigator  Date 
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3. 


Minimal  Risk  Consent  Statement 


The  following  page  represents  the  Minimal  Risk  Consent  Statement  presented  to 
each  participant.  Each  participant  was  required  to  read  and  sign  this  document  prior  to 
participating  in  the  experiment. 
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MINIMAL  RISK  CONSENT  STATEMENT 


NAVAL  POSTGRADUATE  SCHOOL,  MONTEREY,  CA  93943 
MINIMAL  RISK  CONSENT  STATEMENT 

Participant:  VOLUNTARY  CONSENT  TO  BE  A  RESEARCH  PARTICIPANT  IN: 

Virtual  Environment  Interface  Effects  on  Collaboration 

1.  I  have  read,  understand  and  been  provided  "Information  for  Participants"  that  provides  the 
details  of  the  below  acknowledgments. 

2.  I  understand  that  this  project  involves  research.  Ai  explanation  of  the  purposes  of  the 
research,  a  description  of  procedures  to  be  used,  identification  of  experimental  procedures, 
and  the  extended  duration  of  my  participation  have  been  provided  to  me. 

3.  I  understand  that  this  project  does  not  involve  more  than  minimal  risk.  I  have  been  informed 
of  any  reasonably  foreseeable  risks  or  discomforts  to  me. 

4.  I  have  been  informed  of  any  benefits  to  me  or  to  others  that  may  reasonably  be  expected  from 
the  research. 

5.  I  have  signed  a  statement  describing  the  extent  to  which  confidentiality  of  records  identifying 
me  will  be  maintained. 

6.  I  have  been  informed  of  any  compensation  and/or  medical  treatments  available  if  injury 
occurs  and  is  so,  what  they  consist  of,  or  where  further  information  may  be  obtained. 

7.  I  understand  that  my  participation  in  this  project  is  voluntary,  refusal  to  participate  will 
involve  no  penalty  or  loss  of  benefits  to  which  I  am  otherwise  entitled.  I  also  understand  that 
I  may  discontinue  participation  at  any  time  without  penalty  or  loss  of  benefits  to  which  I  am 
otherwise  entitled. 

8.  I  understand  that  the  individual  to  contact  should  I  need  answers  to  pertinent  questions  about 
the  research  is  Krist  Norlander,  Principal  Investigator,  and  about  my  rights  as  a  research 
participant  or  concerning  a  research  related  injury  is  the  Modeling  Virtual  Environments  and 
Simulation  Chairman.  A  full  and  responsive  discussion  of  the  elements  of  this  project  and 
my  consent  has  taken  place. 

Medical  Monitor:  Flight  Surgeon,  Naval  Postgraduate  School 


Signature  of  Volunteer  Date 


Signature  of  Principal  Investigator  Date 
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4. 


In-brief  Statement 


The  following  page  represents  the  In-brief  Statement  presented  to  each 
participant.  Etch  participant  was  required  to  read  this  document  prior  to  commencing  the 
experiment. 
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Collaboration  In-Brief 


You,  as  a  participant  in  this  experiment,  have  been  assigned  to  a  small  collaboration  group  consisting  of 
three  (3)  persons.  Your  assignment  to  this  group  was  predicated  on  the  results  obtained  from  the 
personality  questionnaire  completed  in  the  first  stage  of  the  experiment. 

Experiment  Focus: 

The  focus  of  the  experiment  is  on  collaboration  interaction  within  a  small  group.  For  data  collection 
purposes  your  participation  will  be  recorded  on  video. 

Objective: 

Your  objective  in  this  experiment  is  to  work  with  the  other  participants  in  your  group  to  search,  identify, 
and  locate  as  many  targets  as  possible  within  the  time  period  allotted.  This  task  is  similar  to  a  military 
reconnaissance  mission. 

Tools: 

Each  person  in  your  group  has  a  different  set  of  tools  available  to  assist  in  achieving  your  goal.  The  tools 
available  to  any  one  participant  are  not  enough  to  effectively  solve  the  problem.  The  best  results  will  be 
achieved  through  collaboration  with  the  other  group  members.  It  will  be  advantageous  to  the  group  for 
each  member  to  be  aware  of  the  tools  the  other  members  have  available. 

Collaboration: 

You  can  talk  to  the  other  participants  via  the  audio  headset  provided.  This  headset  includes  a  voice- 
activated  microphone  that  supports  full-duplex  network  communication. 

In  addition  to  the  headset,  each  person  in  your  group  will  be  represented  by  an  avatar  (computer  generated 
soldier).  The  other  participants  will  see  your  avatar  and  you  will  be  able  to  see  theirs.  Your  avatar  has  a 
limited  set  of  interactions  that  will  represent  what  you  are  doing  in  the  virtual  environment.  The  avatar 
can  “stand”,  “walk”,  “run”,  and  “aim  a  rifle”.  The  avatar  state  of  “aiming  your  rifle”  will  be  used  to 
show  the  other  participants  that  you  are  pointing  in  a  given  direction. 

Requirements: 

Throughout  the  experiment  you  will  be  required  to  identify  the  other  group  members  by  one  of  the 
following  names:  “Echo”,  “Kilo”,  “Tango”.  Even  if  you  think  you  know  another  group  member  via 
voice  recognition,  you  are  required  not  to  address  that  person  by  their  real  name. 

The  only  other  requirement  is  that  all  members  of  the  group  must  remain  together  and  agree  on  the 
actions  of  the  group.  Each  member  will  have  a  diagram  for  marking  the  position  of  the  targets  located 
and  identified. 

Experiment  Sequence: 

You  will  participate  in  three  (3)  15-minute  reconnaissance  sessions.  After  each  session,  you  will  be 
required  to  complete  a  short  series  of  questions  about  the  roles  each  member  undertook  within  the  virtual 
environment  and  then  rotate  to  a  new  interaction  station. 

Practice: 

When  you  are  ready,  you  will  be  provided  a  five  (5)  minute  period  to  become  familiar  with  the  navigation 
procedures  within  the  virtual  environment.  A  short  task  list  will  be  provided  to  assist  in  familiarization. 

Please,  ask  any  questions  you  have  prior  to  starting  the  experiment.  During  the  experiment,  the  Principle 
Investigator  will  monitor  the  audio  channel  and  be  available  to  answer  a  limited  set  of  questions. 
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5.  Interim  Questionnaire 

Hie  following  page  represents  the  Participant  Interim- Questionnaire  presented  to 
each  participant  during  the  experiment.  Each  participant  was  required  to  complete  this 
survey  after  completing  the  navigation  task  at  each  workstation. 
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Participant  ID: _  Callsign: _  Run  # _ 

Participant  Interim  Questionnaire 

Please,  answer  the  following  questions  as  honestly  as  possible.  Your  responses  will  be  utilized  to  better  understand 
the  interactions  of  each  member  of  this  collaboration  group  exercise.  All  questions  should  be  answered  referencing 
the  collaboration  experience  you  iust  completed.  Mark  your  responses  on  the  scale  by  placing  an  “X”  at  the 


appropriate  level. 

Least  |— . | - 1 . | - 1 . 1— ~ ^-i - 1— . | - 1 - 1  Most  , 

Identify  how  much  of  the  overall  talking  Was  done  by  each  group  member  (including  yourself). 

ECHO  Least  | — . | - 1 - 1 . | - 1 - 1 - 1— . | - 1 - 1  Most 

KILO  Least  | . | . | - 1 — . | . | . | - 1 - 1 - 1 - 1  Most 

TANGO  Least  | . | . | - 1 . | - 1 - 1 - 1 - 1 - 1 - 1  Most 

Identify  the  amount  of  verbal  feedback  (response  to  another  member)  that  was  provided  by  each  group  member 
(including  yourself). 

ECHO  Least  | - 1 - 1 - 1 . | . | - 1 - 1 . | - 1 - 1  Most 

KILO  Least  | - 1 - 1 - 1 - 1 . | - 1 - 1 . | - 1 - 1  Most 

TANGO  Least  | - 1 - 1— . | . | - 1 - 1 - 1 - 1 - 1 - 1  Most 

Identify  the  quality  (“helpfulness”)  of  the  verbal  feedback  that  was  provided  by  each  group  member  (including 
yourself). 

ECHO  Low  | . | - 1 . | - 1 - 1 - 1 - 1 - 1 - 1 - 1  High 

KILO  Low  | - 1 - 1 - 1 - 1 . | - 1 . | - 1 - 1 - 1  High 

TANGO  Low  | - 1 - 1 . |- . | . | . -| - 1 - 1 - 1 . |  High 

Identify  the  amount  of  input  that  was  provided  by  each  group  member  (including  yourself)  that  appeared  effective  at 
achieving  the  objectives  (communicating,  finding  targets,  and  positioning  targets  on  maps). 

ECHO  Least  | - 1 . | . -| - 1 - 1 - 1 - 1 - 1 . |- . |  Most 

KILO  Least  | - 1 - 1 . -| - 1 . | - 1 - 1 - 1 - 1 - 1  Most 

TANGO  Least  | - 1— . | - 1- . | - 1 - 1 - 1 - 1 - 1 - 1  Most 

Identify  the  amount  of  overall  involvement  by  each  group  member  in  discussions  and  decisions. 

ECHO  Least  | - 1 - 1 - 1 - 1 . | . | . | - 1 . | - 1  Most 

KILO  Least  | . | . | — — — | . | . | - 1 - 1 - 1 - 1 - 1  Most 

TANGO  Least  | - 1 - 1 . | . | . | - 1 . | - 1 - 1 - 1  Most 
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6.  Post  Questionnaire 

The  following  page  presents  the  Participant  Post- Questionnaire  presented  to  each 
participant  after  completing  the  experiment.  Each  participant  was  required  to  complete 
this  survey  after  completing  the  navigation  tasks  at  all  of  the  three  workstations. 
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Participant  ID: 


Participant  Post  Questionnaire 

Please,  answer  the  following  questions  as  fully  and  honestly  as  possible.  Your  responses  will  be  utilized  to  improve 
future  research  in  Distributed  Collaboration.  When  answering  a  question  regarding  interface,  each  interface  should 
be  identified  by  the  following  terms: 

CAVE  -  Large  3  screen  display 

PV230  -  Small  3  screen  display 

Single  -  Small  single  screen  display 


1.  Which  interface  provided  you  the  most  enjoyable  experience?  Why? 


2.  What  was  your  preferred  interface?  Why? 


3.  Which,  if  any,  of  the  interfaces  did  you  feel  interfered  the  most  with  your  ability  to  meet  the  objectives 
(communicate,  navigate,  locate  and  identify  targets)? 


4.  If  applicable,  how  did  the  interface  identified  in  above  (question  3)  interfere? 


5.  Which,  if  any,  of  the  interfaces  made  you  feel  “less  a  part”  of  the  group?  Why? 


6.  Did  you  experience  difficulties  verbally  communicating  yourself  over  the  networked  communication  system? 

7.  If  applicable,  describe  the  verbal  communication  conflicts  experienced?  Why? 


8.  Did  you  find  the  “pointing”  capability  useful  to  group  interaction?  Why  or  why  not? 


9.  Overall,  which  group  member  would  you  identify  as  the  leader?  Why? 

ECHO  Least  | - 1 . | . | - 1 . | - 1 - 1 - 1 - 1 - 1  Most 

KILO  Least  | . | . | - 1 - 1 - 1 . | - 1 . | - 1 - 1  Most 

TANGO  Least  | - 1 - 1 - 1 . | - 1 - 1 - 1 - 1 - 1- - 1  Most 
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APPENDIX  C.  RAW  DATA 


A.  GROUP  1  -  INTERIM-  AND  POST-QUESTIONNAIRE  SCORES 
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A 

B 

C 

C 

A 

B 

B 

C 
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16 
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9.4 

9.5 
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53 

2.6 
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5.5 

6.5 

6.5 

5.5 
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6.5 

6.4 

59 

2.4 

2.4 

3.5 

6.5 

5.6 

4.5 

4.6 

6.4 

4.5 

Feedback  Quantity 

16 

3.4 

2.5 

9.5 

3.5 

9.5 

5.5 

3.5 

2.5 

3.4 

53 

3.6 

4.5 

5.5 

4.5 

5.6 

5.5 

6.5 

6.4 

5.5 

59 

1.5 

1.5 

2.5 

7.5 

6.4 

4.5 

6.5 

6.5 

5.4 

Feedback  Quality 

16 

3.5 

4.4 

8.3 

8.5 

9.3 

7.6 

4.4 

5.4 

6.6 

53 

0.5 

4.5 

5.5 

0.5 

6.5 

6.4 

0.5 

6.5 

6.4 

59 

0.5 

1.5 

1.5 

7.5 

6.4 

4.5 

4.5 

7.4 

4.5 

Effective  Input 

16 

6.5 

5.7 

8.5 

8.6 

6.4 

6.3 

6.5 

7.5 

5.4 

53 

2.4 

4.5 

5.5 

6.4 

5.5 

5.5 

5.6 

6.4 

5.5 

59 

0.5 

1.5 

2.5 

4.5 

5.5 

4.5 

4.5 

5.5 

4.5 

Overall  Involvment 

16 

5.6 

1.5 

8.4 

8.6 

7.6 

7.6 

5.6 

3.5 

5.5 

53 

3.4 

4.5 
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6.4 

5.5 

5.5 

6.4 

5.5 

5.5 

59 

1.5 

0.6 

4.5 

7.3 

5.4 

5.5 

6.5 

7.5 

5.5 

Overall  Leader 

16 

53 

59 

16 

5.6 

8.4 

5.5 

53 

3.5 

6.4 

7.4 

59 

2.5 

8.6 

7.5 

Toolset 

Elements 

A 

Compass  and  Satellite  Image 

Key  Feature  Location  with  Photos 
Compass  Rose  and  Contour  Map 

B 

C 

63 
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GROUP  2  -  INTERIM-  AND  POST-QUESTIONNAIRE  SCORES 


Participant  ID 

54 

40 

34 

Emergent  Leader  Traits 
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No 

No 

Run  # 

1 

2 

3 

1 

2 

3 

1 

2 

3 

Workstation  Interface 

Cave 

PV230 

Single 

PV230 

Single 

Cave 

Single 

Cave 

PV230 

Tool  Set 

C 

A 

B 

A 

B 

C 

B 

C 

A 

Amount  of  Talking 

54 

4.6 

5.0 

5.0 

4.6 

5.0 

5.0 

4.6 

5.0 

5.0 

40 

5.6 

6.6 

7.4 

6.6 

8.5 

8.4 

7.4 

7.5 

7.4  ! 

34 

5.0 

8.0 

8.0 

7.0 

8.0 

8.0 

3.0 

8.0 

4.0 

Feedback  Quantity 

54 

4.7 

5.0 

5.0 

4.7 

5.0 

5.0 

4.7 

5.0 

5.0 

40 

5.4 

7.4 

7.5 

5.4 

8.5 

8.4 

5.4 

7.3 

7.4 

34 

7.0 

8.0 

8.0  1 

7.0 

8.0 

8.0 

6.0 

7.0 

8.0 

Feedback  Quality 

54 

5.1 

5.0 

5.0 

2.9 

5.0 

5.0 

2.7 

5.0 

5.0 

40 

6.5 

6.5 

7.5 

6.5 

7.5 

8.5 

6.5 

8.4 

7.3 

34 

7.0 

8.0 

8.0 

7.0 

8.0 

7.0 

7.0 

8.0 

7.0 

Effective  Input 

54 

3.6 

5.0 

5.0 

2.6 

5.0 

5.0 

3.8 

5.0 

5.0 

40 

4.4 

6.3 

4.5 

4.4 

8.5 

4.5 

4.4 

7.4 

4.5 

34 

9.0 

9.0 

7.0 

9.0 

9.0 

7.0 

9.0 

8.0 

7.0 

Overall  Involvment 

54 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

40 

6.5 

7.4 

7.5 

6.5 

8.4 

7.5 

6.5 

7.4 

7.5 

34 

9.0 

9.0 

9.0 

9.0 

9.0 

9.0 

9.0 

9.0 

9.0 

Overall  Leader 

54 

40 

34 

54 

5.0 

5.0 

5.0 

40 

5.50 

6.50 

7.50 

34 

10.00 

9.00 

8.00 

Toolset 

Elements 

A 

Compass  and  Satellite  Image 

Key  Feature  Location  with  Photos 
Compass  Rose  and  Contour  Map 

B 

C 

64 


c 


GROUP  3  -  INTERIM-  AND  POST-QUESTIONNAIRE  SCORES 


Participant  ID 

35 

91 

41 

Emergent  Leader  Traits 

No 

Yes 

No 

Run  # 

1 

2 

3 

1 

2 

3 

1 

2 

3 

Workstation  Interface 

Cave 

Single 

PV230 

PV230 

Cave 

Single 

Single 

PV230 

Cave 

Tool  Set 

B 

A 

C 

C 

B 

A 

A 

C 

B 

Amount  of  Talking 

35 

4.4 

5.3 

4.5 

7.5 

8.4 

7.5 

2.5 

0.6 

6.4 

91 

4.4 

4.5 

4.5 

6.2 

4.5 

5.3 

2.3 

0.0 

6.4 

41 

5.5 

6.5 

3.5 

9.3 

9.5 

6.5 

2.3 

5.6 

7.6 

Feedback  Quantity 

35 

7.4 

7.5 

4.5 

3.5 

7.5 

6.5 

6.5 

0.4 

7.4 

91 

7.2 

4.5 

5.5 

1.6 

4.5 

4.5 

3.4 

0.0 

6.5 

41 

4.3 

5.5 

4.6 

3.5 

4.5 

5.5 

6.5 

6.3 

6.7 

Feedback  Quality 

35 

7.4 

7.6 

4.4 

5.4 

6.5 

6.5 

1.4 

0.5 

8.4 

91 

8.5 

4.5 

4.5 

8.5 

4.5 

6.5 

2.5 

0.0 

6.5 

41 

6.5 

5.5 

4.5 

7.3 

7.4 

7.6 

3.5 

3.5 

7.7 

Effective  Input 

35 

5.5 

7.5 

5.6 

6.6 

6.5 

5.5 

1.5 

0.5 

8.5 

91 

5.5 

4.5 

4.5 

7.5 

4.5 

6.5 

2.5 

0.0 

6.5 

41 

8.5 

5.5 

3.5 

6.5 

8.3 

7.6 

3.5 

2.5 

8.4 

Overall  Involvment 

35 

5.4 

7.5 

5.4 

7.5 

7.6 

7.6 

1.4 

0.5 

5.4 

91 

7.5 

4.5 

6.6 

7.5 

4.5 

6.5 

3.5 

0.0 

5.5 

41 

7.3 

6.5 

4.5 

8.4 

7.5 

6.5 

3.4 

3.5 

8.3 

Overall  Leader 

35 

91 

41 

35 

5.3 

7.4 

3.4 

91 

0.0 

6.5 

0.0 

41 

0.0 

10.0 

0.0 

Toolset 

Elements 

A 

Compass  and  Satellite  Image 

Key  Feature  Location  with  Photos 
Compass  Rose  and  Contour  Map 

B 

C 

65 


D 


GROUP  4  -  INTERIM-  AND  POST-QUESTIONNAIRE  SCORES 


Participant  ID 

74 

81 

65 

Emergent  Leader  Traits 

No 

No 

Yes 

Run  # 

1 

2 

3 

1 

2 

3 

1 

2 

3  ! 

Workstation  Interface 

Cave 
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PV230 

PV230 

Cave 

Single 
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PV230 

Cave 

Tool  Set 

A 

C 

B 

B 

A 

C 

C 

B 

A 

Amount  of  Talking 

74 

0.5 

0.5 

3.5 

8.5 

9.5 

9.3 

8.5 

9.5 

9.3 

81 

2.5 

1.4 

1.0 

8.0 

6.8 

9.0 

6.0 

7.6 

7.5 

65 

4.4 

2.5 

3.4 

9.5 

8.3 

7.6 

7.5 

8.2 

8.5 

Feedback  Quantity 

74 

2.5 

3.5 

9.5 

7.4 

5.5 

9.4 
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5.5 
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81 

2.6 

2.1 
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4.8 
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6.8 

7.5 
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65 

6.3 

1.4 

4.4 

7.4 
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7.5 
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Feedback  Quality 

74 
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7.5 
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81 
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65 
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0.5 

5.4 
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5.3 

6.5 

2.4 

5.3 

6.4 

Effective  Input 

74 

7.5 

0.6 

7.5 

7.5 

5.5 

6.4 

7.5 

5.5 

6.4 

81 

1.9 

3.6 

3.3 

8.0 

8.3 

7.6 

8.2 

7.3 

7.5  ! 

65 

3.4 

0.6 

4.6 

4.4 
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6.4 

4.4 

7.3 

6.3 

Overall  Involvment 

74 

0.5 
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2.4 

9.4 

9.5 

9.4 

9.4 
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9.4 

81 

3.4 

0.4 

0.3 

9.8 

6.5 

7.3 

8.3 

7.9 

7.2 

65 

3.5 

2.4 

4.5 

8.5 

7.5 

7.5 

6.5 

8.5 

8.4 

Overall  Leader 

74 

81 

65  | 

74 

0.6 

9.4 

8.5 

81 

0.4 

8.0 

6.7 

65 

4.3 

7.5 

8.5 

Toolset 

Elements 

A 

Compass  and  Satellite  Image 

Key  Feature  Location  with  Photos 
Compass  Rose  and  Contour  Map 

B 

C 

66 


E. 


GROUP  5  -  INTERIM-  AND  POST-QUESTIONNAIRE  SCORES 


Participant  ID 

92 

85 

77 

Emergent  Leader  Traits 

No 

No 

Yes 

Run  # 

1 

2 

3 

1 

2 

3 

1 

2 

3  ! 

Workstation  Interface 

Cave 
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PV230 

PV230 

Cave 
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PV230 

Cave 

Tool  Set 

C 

B 

A 

A 

C 

B 

B 

A 

C 

Amount  of  Talking 

92 

5.5 

4.0 

2.0 

5.5 

4.0 

2.9 

5.5 

4.0 

3.1 

85 

7.5 

6.5 

6.4 

3.5 

6.5 

7.5 

6.5 

6.5 

5.6 

77 

4.8 

5.4 

5.2 

3.6 

5.4 

5.2 

6.4 

5.0 

5.8 

Feedback  Quantity 

92 

5.0 

2.0 

3.1 

5.0 

2.9 
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3.1 

85 
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6.5 

4.6 

6.5 

2.4 

7.5 

6.6 

5.4 
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77 

4.8 

5.9 

5.7 
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5.0 

5.8 

4.9 

5.0 

5.2 

Feedback  Quality 

92 

1.0 

1.0 

3.0 

1.0 

1.0 

3.0 

1.0 

1.0 

3.1 

85 

7.5 

3.5 

7.3 

6.5 

3.5 

8.6 

5.5 

3.5 

5.5 

77 

5.8 

5.8 

5.4 

5.8 

4.5 

5.3 

5.8 

4.1 

5.7 

Effective  Input 

92 

3.0 

2.0 

3.0 

3.0 

2.0 

3.0 

3.0 

2.0 

3.0 

85 

5.5 

4.5 

6.3 

2.6 

4.5 

7.6 

5.5 

4.5 

5.5  ! 

77 

5.1 
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5.8 

5.0 

4.8 
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5.0 

4.2 

5.8 

Overall  Involvment 

92 

5.0 

3.0 

2.0 

5.0 

3.9 

2.9 

5.0 

4.0 

3.1  i 

85 

6.6 

7.5 

6.4 

6.5 

7.6 

7.5 
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6.4 

77 

5.0 

5.2 

5.5 

5.0 

5.2 

4.6 

5.7 

5.3 

5.3 

Overall  Leader 

92 

85 

77  | 

92 

3.0 

3.0 

3.0 

85 

4.5 

4.5 

4.5 

77 

5.3 

5.2 

5.3 

Toolset 

Elements 

A 

Compass  and  Satellite  Image 

Key  Feature  Location  with  Photos 
Compass  Rose  and  Contour  Map 

B 

C 

67 


F. 


GROUP  6  -  INTERIM-  AND  POST-QUESTIONNAIRE  SCORES 


Participant  ID 

52 

64 

48 

Emergent  Leader  Traits 

No 

Yes 

No 

Run  # 

1 

2 

3 

1 

2 

3 

1 

2 

3  ! 

Workstation  Interface 

Cave 

Single 

PV230 

PV230 

Cave 

Single 

Single 

PV230 

Cave 

Tool  Set 

B 

A 

C 

C 

B 

A 

A 

C 

B 

Amount  of  Talking 

52 

2.0 

4.7 

4.6 

3.3 

5.4 

6.5 

5.2 

4.5 

4.5 

64 

3.0 

3.0 

4.0 

8.0 

10.0 

9.0 

8.0 

8.0 

9.0 

48 

5.0 

4.0 

6.0 

6.0 

6.0 

6.0 

7.0 

5.0 

6.0 

Feedback  Quantity 

52 

2.8 

6.2 

5.8 

3.5 

4.6 

3.8 

3.5 

6.4 

4.2 

64 

4.0 

5.0 

5.0 

8.0 

7.0 

8.0 

7.0 

8.0 

8.0 

48 

5.0 

5.0 

6.0 

7.0 

5.0 

6.0 

6.0 

5.0 

6.0 

Feedback  Quality 

52 

3.3 

7.3 

5.7 

4.7 

4.7 

4.3 

4.8 

6.6 

6.5 

64 

3.0 

6.0 

4.0 

6.0 

6.0 

10.0 

8.0 

6.0 

9.9 

48 

6.0 

5.0 

6.0 

6.0 

5.0 

6.0 

6.0 

5.0 

6.0 

Effective  Input 

52 

5.1 

5.9 

5.6 

5.9 

4.6 

6.9 

5.1 

7.0 

6.1 

64 

3.0 

5.0 

5.0 

6.0 

9.0 

9.0 

7.0 

7.0 

9.o  : 

48 

6.0 

5.0 

6.0 

6.0 

6.0 

6.0 

6.0 

5.0 

6.0 

Overall  Involvment 

52 

3.2 

6.0 

5.5 

5.1 

4.5 

7.2 

5.1 

6.7 

6.3  i 

64 

3.0 

3.0 

3.9 

8.0 

10.0 

8.9 

8.0 

8.0 

8.9  i 

48 

6.0 

5.0 

6.0 

6.0 

6.0 

6.0 

6.0 

6.0 

6.0 

Overall  Leader 

52 

64 

48 

52 

4.9 

5.2 

6.7 

64 

3.0 

10.0 

9.0 

48 

5.0 

6.0 

5.0 

Toolset 

Elements 

A 

Compass  and  Satellite  Image 

Key  Feature  Location  with  Photos 
Compass  Rose  and  Contour  Map 

B 

C 
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G.  PARTICIPANT  PERSONALITY  TRAIT  SCORES 

For  privacy  concerns,  the  data  received  from  each  participant  was  not  published 
with  this  document.  For  future  research  requiring  the  use  of  this  data,  the  Principle 
Investigator,  LT  Krist  Norlander,  USNR,  or  the  Thesis  Advisor,  Dr.  Rudolph  P.  Darken, 
should  be  contacted  at  The  MOVES  Institute,  Naval  Postgraduate  School,  Monterey, 
California  for  official  release  of  this  information. 
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